1. Field of the Invention
The present invention relates to a video coding device, and more particularly to a device for coding and transmitting moving pictures.
2. Description of the Background Art
(First Conventional Example)
Conventional video compression coding systems for moving picture communication at a low bit rate include ITU Recommendation H.261. FIG. 18 is a block diagram showing the structure of a video coding device for realizing the video coding system recommended in H.261. Structures and operations of parts of this first conventional example will be described below.
In FIG. 18, a video coding device 90 includes a video input portion 11, a coding device 12, a transmission control device 17 and an output device 18. The transmission control device 17 includes a transmission buffer 171 and a transmission control portion 172.
The video input portion 11, composed of a camera etc., performs input of video. The coding device 12 codes the video inputted from the video input portion 11 according to the video coding method recommended in H.261. The transmission buffer 171 stores the video data coded in the coding device 12. The transmission control portion 172 transmits the coded video data stored in the transmission buffer 171 to a video receiving device 1000 at a predetermined transmission rate through the output device 18.
FIG. 19 is a block diagram showing more detailed structure of the coding device 12 in FIG. 18. In FIG. 19, the coding device 12 includes a video input device 100, a coding control portion 101, a SW (switch) 102, a scan conversion portion 103, a differential portion 104, a quantization portion 105, a motion vector detecting portion 106, a motion compensation portion 107, a frame memory 108, a inverse quantization portion 109, an addition portion 10a, a multiplexer 10b and a code output device 10c. 
The SW 102 switches whether to code or not a video frame inputted from the video input portion 11 (that is, frame dropping or not). The coding control portion 101 carries out frame dropping control for controlling whether the video frame inputted at the video input portion 11 is to be coded and transmitted, or to be dropped without coded and quantization control for determining the most suitable quantization step size from both the reproducibility of the motion and the video quality. The scan conversion portion 103 scan-converts the moving picture data inputted from the video input device 100 and outputs it as data in block unit composed of a plurality of pixels. The motion vector detecting portion 106 detects the motion vector of the data block outputted from the scan conversion portion 103. The frame memory 108 stores local decoded moving picture data one frame before. The motion compensation portion 107 retrieves a prediction value from the moving picture data in the frame memory 108 on the basis of the motion vector information which is an output of the motion vector detecting portion 106 and outputs the same.
The differential portion 104 calculates the difference between the data block from the scan conversion portion 103 and the prediction value from the motion compensation portion 107 to output a prediction error. The quantization portion 105 quantizes the prediction error from the differential portion 104. The inverse quantization portion 109 inversely quantizes the prediction error quantized in the quantization portion 105. The addition portion 10a adds the prediction error inversely quantized in the inverse quantization portion 109 and the prediction value from the motion compensation portion 107 to calculate decode moving picture data and stores it in the frame memory 108. The multiplexer 10b multiplexes the prediction error quantized in the quantization portion 105 and the motion vector detected in the motion vector detecting portion 106.
Coding operation of the conventional video coding device constructed as described above will now be described. The frame dropping control and the quantization control by the coding control portion 101 will be described later.
The moving picture frame inputted from the video input portion 11 is inputted to the coding,device 12. In the coding device 12, the moving picture signal inputted from the video input device 100 is scan-converted by the scan conversion portion 103 and then inputted to the motion vector detecting portion 106 for each block formed of a plurality of pixels. The motion vector detecting portion 106 compares the local decoded video data one frame before stored in the frame memory 108 and the block inputted from the scan conversion portion 103 to obtain motion vector and outputs it to the motion compensation portion 107 and the multiplexer 10b. The motion compensation portion 107 finds a corresponding prediction value from the video data in the frame memory 108 on the basis of the inputted motion vector information and outputs it to the differential portion 104 and the addition portion 10a. 
The differential portion 104 calculates a difference between the block outputted from the scan conversion portion 103 and the prediction value outputted from the motion compensation portion 107 to find a prediction error and outputs it to the quantization portion 105. The quantization portion 105 quantizes this prediction error with a quantization step size determined in the coding control portion 101 and outputs it to the multiplexer 10b and the inverse quantization portion 109. The inverse quantization portion 109 inversely quantizes the prediction error and outputs it to the addition portion 10a. The addition portion 10a adds the inversely quantized prediction error and the prediction value outputted from the motion compensation portion 107 to decode the video data and stores it in the frame memory 108. The quantized prediction error and the motion vector are multiplexed in the multiplexer 10b and stored in the transmission buffer 171 through the code output device 10c. The transmission control portion 172 transmits the coded video data in the transmission buffer 171 at a predetermined transmission rate to the video receiving device 1000 through the output device 18.
Next, the frame dropping control performed in the coding control portion 101 will be described. Conventionally, as the frame dropping control system, the system of determining whether frame dropping is required or not on the basis of comparison between the accumulated amount in the transmission buffer and a determination threshold is proposed. (Refer to Yasuhiro Takishima et al. xe2x80x9cA Study on Control Method for Low Bit Rate Video Coding,xe2x80x9d The Institute of Electronics, Information and Communication Engineers, Spring National Meeting, D-311, pp.7-63, 1990.) The coding control portion 101 performs frame dropping control using the technique disclosed in this paper.
FIG. 20 is a sequence diagram showing operation of the conventional frame dropping control system. Referring to FIG. 20, the coding control portion 101 carries out coding of video frame if the accumulated amount in the transmission buffer 171 is smaller than a certain threshold at the start of coding (that is, it connects the devices 1-2 of the SW 102) and if it is larger than the certain threshold, it does not code that video frame, but drops the frame. (That is to say, it connects the devices 1-3 of the SW 102.) In FIG. 20, at the points (T1, T3) at which coding of the frame 2 and the frame 4 is started, the accumulated amount in the transmission buffer 171 exceeds the threshold, so that the frames 2 and 4 are dropped.
Next, the quantization control made in the coding control portion 101 will be described. Conventionally, the coding control system of determining a quantization step size for a coded frame using a coding rate of already coded video frames is proposed. (Refer to Japanese Patent Laying-Open No.7-107482.) The coding control portion 101 adopts the coding control system disclosed in this reference to achieve quantization control to enhance video quality as far as reproducibility of motion is not deteriorated. The conventional coding control system for determining the quantization step size for a coded object frame using the coding rate will now be described.
FIG. 21 shows an example of coding control in the conventional coding control system. In FIG. 21, lines 101A-101D show coding rate-quantization accuracy characteristics, the line 102 shows the upper limit values of the visually most suitable coding rates, the line 103 shows the lower limit values of the visually most suitable coding rates, 104A shows the first quantization accuracy, 104B shows the second quantization accuracy, 104C shows the third quantization accuracy, the character xe2x97xaf shows the visually most suitable operating points, and the character ∘ shows transient operating points. The vertical axis shows the coding rate and the horizontal axis shows the quantization accuracy. On the horizontal axis, the quantization accuracy becomes lower and spacial distortion of the displayed video becomes larger as it gets to the right. On the vertical axis, the coding rate becomes lower and the possibility of frame dropping becomes higher as it gets down. That is to say, the timewise distortion of the displayed video becomes larger.
Here, the coding rate is a value showing the ratio of coding video frames. If all video frames inputted from the video input portion 11 in FIG. 18 are coded, the coding rate is 1.0, and if they are coded once for every two frames (one frame dropping), then the coding rate is 0.5.
Next, the coding rate-quantization accuracy characteristics will be described briefly. In moving picture coding, the space resolution (quantization accuracy) and the time resolution (coding rate) are in the trade-off relation and there are several combinations. The set of the combinations of the space resolution and the time resolution is referred to as coding rate-quantization accuracy characteristics herein. The coding rate-quantization accuracy characteristics vary according to the magnitude timewise of motion of the input video and spacial frequency components. The curve 101 representing the coding rate-quantization accuracy characteristics moves down to the right on the coordinates shown in FIG. 21 if the displayed video includes large motion or fine pattern and it moves up to the left if motion is small.
The upper limit curve 102 is a curve obtained by connecting points representing upper limit values of the most suitable coding rates on the curves 101 and the lower limit curve 103 is a curve obtained by connecting the points representing the lower limit values of the most suitable coding rates on the curves 101. That is to say, in a visually most suitable region assumed on the curves 101, the upper limit curve 102 of the most suitable coding rate is obtained by connecting the points of the largest coding rates and the lower limit curve 103 of the most suitable coding rate is obtained by connecting the points of the smallest coding rates.
When the coding rate resulting from coding with the first quantization accuracy 104A is larger than the upper limit value 102 of the coding rate most suitable in view of visual properties corresponding to the first quantization accuracy, the coding control portion 101 performs next coding with the second quantization accuracy 104B more accurate than the first quantization accuracy. If it is smaller than the lower limit value 103, it performs next coding with the third quantization accuracy 104C less accurate than the first quantization accuracy. When the coding rate resulting from coding with the first quantization accuracy 104A is in the region, it performs next coding with the first quantization accuracy 104A. After that, the coding control portion 101 similarly determines the quantization accuracy for next coding according to the relation in magnitude between a coding rate resulting from coding with a certain quantization accuracy and the region of coding rates most suitable in view of visual properties corresponding to that quantization accuracy.
Next, referring to FIG. 22, conventional coding control in the case in which a rapid motion of an object occurs will be described. In FIG. 22, lines 2231 to 2233 show the coding rate-quantization accuracy characteristics, lines 2211 to 2214 show the range of the visually most suitable coding rates, 2221 to 2225 show the quantization accuracy, the character xe2x97xaf shows visually most suitable operating points, and the characters ∘ shows transient operating points. The vertical axis shows the coding rate and the horizontal axis shows the quantization accuracy. On the horizontal axis, the quantization accuracy becomes lower and the spatial distortion of the displayed video becomes larger as it gets to the right. On the vertical axis, the coding rate becomes lower and the possibility of frame dropping becomes higher as it gets down. That is to say, the timewise distortion of the displayed video becomes larger.
For example, if such a small motion of the object that the coding rate-quantization accuracy characteristic moves from 2231 to 2232 occurs when operating at the operating point A, the frame in which the small motion of the object occurs is coded with the quantization accuracy 2221 and the operating point shifts to the operating point B. The coding rate at this time is between the upper limit and the lower limit of coding rate for the quantization accuracy 2221, so that the next video frame is also coded with the quantization accuracy 2221.
If such a rapid movement of the object that the coding rate-quantization accuracy characteristic moves from 2231 to 2233 occurs, the frame in which the rapid movement of the object occurs is coded with the quantization accuracy 2221 and the operating point moves to the operating point C. The coding rate at this time is under the lower limit of the coding rate for the quantization accuracy 2221, so that the next video frame is coded with the quantization accuracy 2222 and the operating point shifts to the operating point D. The coding rate of the video frame coded with the quantization accuracy 2222 is under the lower limit of the coding rate for the quantization accuracy 2222, so that the next video frame is coded with the quantization accuracy 2223 and the operating point moves to the operating point E. Subsequently, the same operation is repeated until the coding rate enters the visually most suitable region (until the quantization accuracy 2225 is achieved.)
The quantization accuracies in FIG. 21 and FIG. 22 correspond to the quantization step size. That is to say, the quantization step size is smaller when the quantization accuracy is good (high) and the quantization step size is larger when the quantization accuracy is bad (low).
(Second Conventional Example)
Structure and operation of a conventional video coding device with video data length and error correcting code mode variable according to the conditions of communication will be described below.
Conventionally, such a low bit rate coding device for television signals is proposed as enlarges the video data length and adds an error correcting code with a low error correcting capability when the communication condition is good, and on the other hand, when the communication condition becomes bad, shortens the video data length and adds an error correcting code with high error correcting capability. (Refer to Japanese Patent Laying-Open No.3-78383.) FIG. 23 is a block diagram showing the structure of the low bit rate coding device for television signals disclosed in this reference. In FIG. 23, the low bit rate coding device 130 includes a low-pass filter 131, a sub-sampling circuit 132, a predictive coding circuit 133, a buffer memory 134, a device control circuit 137, a variable length coding circuit 135 and an error correction coding circuit 136.
Operation of the low bit rate coding device for television signals constructed as shown in FIG. 23 will be described below. The output of the variable length coding circuit 135 is inputted to the error correction coding circuit 136. The error correction coding circuit 136 adds an error correcting code to the variable length coded video data. The error correcting code length is switched by a normal/fall back mode selection signal. That is to say, the error correcting code is shortened in the normal mode (that is, being an error correcting code with low error correcting capability), and lengthened in the fall back mode (that is, being an error correcting code with high error correcting capability). The predictive coding circuit 133 makes coding in the normal mode so that the video data length becomes longer (that is to say, it performs quantization with a quantization step size with a small quantization step size), and it performs coding in the fall back mode so that the video data length becomes shorter (that is to say, it performs quantization with a quantization step size with a large quantization step size). When the communication condition becomes bad, the operation mode is switched from the normal mode to the fall back mode. In the fall back mode, in which the video data is shorter than that in the normal mode, the error correcting code can be made longer. Thus, the error rate can be suppressed low without enlarging the entire code length (video data + error correcting code.)
(Third Conventional Example)
A conventional video coding device which performs hierarchical coding with coded regions divided operates as follows.
That is to say, a priority area and a non-priority area are previously determined in a coded area of video captured from a camera. In the video coding portion, coding of the priority area is made with a quantization accuracy higher than the quantization accuracy for the non-priority area.
(Problems of the First Conventional Example)
The video coding device according to the aforementioned first conventional example has the four problems described below.
[First Problem]
The video coding device according to the first conventional example calculates the coding rate on the basis of the amount of generated information coded one frame before and the transmission rate and determines the quantization step size for the coded object frame using that calculated coding rate. Therefore, if the object suddenly moves, the coding may be performed with an unsuitable coding value which does not match the coded frame. As a result, the amount of generated information rapidly increases and the delay time before displayed on the receiving side increases, then a moving picture with unnatural motion will be displayed on the receiving side. Furthermore, because of the increase in amount of the generated information, heavy frame dropping will occur to reduce the interrelation between frames. This will lead to the vicious circle that the amount of generated information increases also for the video frame coded next.
The relation between the delay time and the frame jump caused by an increase in amount of generated information will be described referring to FIG. 24. FIG. 24 shows the delay time and the frame jump in the conventional video coding device. Before the point P1 at which a sudden movement of the object occurs, the movement of the object is small and thus the prediction error between frames is small. As a result, coding is made with a quantization step size with a small quantization step size, and the delay time does not exceed 440 msec. At the point P1 at which the sudden movement of the object occurs, coding is made with a quantization step size with a small quantization step size in spite of the increase in prediction error between frames, resulting in the delay time of 1275 msec and frame jump of 42 frames. The frame coded after the point P1 has low inter-frame relation because of the jump of 42 frames, so that the prediction error between frames increases, the amount of generated information increases, and thus the delay time increases.
[Second Problem]
In the video coding device according to the first conventional example, the operating point moves in steps when the object suddenly moves, therefore coding is carried out with an unsuitable quantization step size which does not match the coded object frame until the quantization step size matching the coded object frame is selected. As a result, the movement may be unnatural or the quality of the moving picture will be deteriorated until the quantization step size matching the coded object frame is selected.
To solve the first and second problems, such three devices or circuits as shown below have been proposed conventionally.
(1) A video signal coding device which predicts the amount of generated information on the basis of the sum of difference absolute values between frames, or the dynamic range to determine a quantization step size corresponding to the features of the coded object frame. (Refer to Japanese Patent Laying-Open No.5-336513.)
(2) A buffer memory control circuit of a video coding device which determines whether motion is large or small on the basis of the size of motion vector, and if the motion vector is large, performs coding with a quantization step size with a large quantization step size, and if the motion vector is small, it performs coding with a quantization step size with a small quantization step size. (Refer to Japanese Patent Laying-Open No.5-130587.)
(3) A video coding device which includes a plurality of quantization portions and performs quantization with quantization step sizes changed to determine a quantization step size with good vision. (Refer to Japanese Patent Laying-Open No.4-323961.)
However, the above three devices or circuits produce such new problems as follows. That is to say, in the video signal coding device of (1), the prediction often fails, and then the same problem mentioned in the first problem arises. While the buffer memory control circuit of (2) determines the quantization step size on the basis of the size of the motion vector, the amount of generated information does not necessarily increase if the motion vector is large, for the prediction error between frames can be small if the motion vector follows the motion of the object. Accordingly, coding is made with a quantization step size with a large quantization step size even if the amount of generated information is small though the motion vector is large, and then the video quality is deteriorated more than needed. The video coding device of (3) must include a plurality of quantization portions, so that the circuit scale becomes large and the cost increases.
[Third Problem]
In the video coding device of the first conventional example, the coding rate is calculated using the same transmission rate regardless of the communication conditions, so that the coding may be carried out with an unsuitable quantization step size which does not match the communication conditions. A calculation expression for the coding rate in the conventional quantization control follows.
Coding Rate=(LO/S)/RO
Note that in the expression above;
LO: transmission rate (constant),
S: the number of frame samplings per second, and
RO: the amount of generated information of the frame coded before (one frame before.) As can be seen from the expression above, when a transmission error occurs due to deterioration of the communication condition, the video coding device on the transmission side makes retransmission and the transmission throughput changes, then an unsuitable quantization step size not matching the communication condition is determined.
To solve the third problem above, the applicant of the present invention has proposed in Japanese Patent Application No.6-104007 a coding control method for making correction by using, as the transmission rate LO for calculating a coding rate for an n-th frame, an average communication rate from the start of coding the n-th frame to the start of the next coding.
However, according to xe2x80x9cImprovement Techniques of Picture Coding Quality Deteriorations for Mobile Communicationsxe2x80x9d (Hisashi IBARAKI, et al., The Institute of Image Electronics Engineers of Japan, vol.23, No.5 (1994), pp.445-453), the Doppler frequency is 5 Hz at the walking rate of a human (3 Km/h) in the frequency band of the so-called PHS (Personal Handy-phone System), and then the number of times of one-digit deterioration of transmission error is 3.5 times/second and the average continuation time is 20 msec. That is to say, to recognize the burst errors occurring in the PHS on average, it is necessary to measure the average in the time width of at least 300 msec. While the number of video frames deteriorates below 30 pieces/second when a moving picture is transmitted at a low bit rate, even if it deteriorates to 5 pieces/second, the time from when the n-th frame is coded until when the next frame is coded is 200 msec. Then correcting the coding rate using the average communication rate therebetween will cause the problem that the burst error is dominant and correction goes to extremes.
Further, in the coding control method, the frame to be coded is determined by the coding rate of the frame previously coded. In this case, while there is no problem when the change of the communication rate is mild, when the communication rate decreases rapidly, the correction of the coding rate cannot overtake and, in reality, the next frame is disadvantageously coded before the data in the transmission buffer decreases, resulting in overflow of the buffer.
Moreover, a retransmission control method by Go-Back-N is assumed here, and the average throughput is calculated by the transmittal confirmation bit number of the frames between when the coding of the n-th frame is started and when the next coding is started.
In the retransmission control method by Go-Back-N, however, after transmission of eight frames, for example, transmittal confirmation or retransmission request is waited on the receiving side, and in the case of retransmission request, all frames after the requested frame are retransmitted and the retransmission is repeated until transmission is confirmed. In the retransmission control method by Go-Back-N, waiting time increases when the delay time of the transmission line is large, resulting in the transmission efficiency is extremely degraded. Therefore, a retransmission control method by Selective Reject is conventionally considered. In the retransmission control method by Selective Reject, the transmission side transmits, for example, up to 32 frames without waiting transmittal confirmation, and the receiving side selectively transmits reject to the frame in which an error is detected. When receiving reject, the transmission side immediately retransmits the frame. The transmittal confirmation is not performed in the retransmission control method by Selective Reject.
[Fourth Problem]
In the quantization control method performed in the coding control portion 101 of the video coding device according to the first conventional example, when motion of the displayed video is too large or the transmission throughput considerably decreases due to a large amount of transmission errors, both the space resolution (quantization accuracy) and the time resolution (coding rate) may exceed the permissible limits to cause visually unpermissible deterioration, though they may be the most suitable in selectable trade-off combinations. The coding rate-quantization accuracy characteristics are determined by the extent of the incompressible non-redundancy of the given video, such as the quantity of movement and fineness of the pattern, and the transmission rate for transmitting the coded data. These factors are something to be given, but not something to select. If selectable quantization accuracies only are considered as parameters, it is the most suitable to operate in the region between the upper limit curve 102 and the lower limit curve 103 under the given conditions. However, the xe2x80x9cmost suitablexe2x80x9d here is merely relatively best under the given conditions, but not absolutely good. Absolutely, both the quantization accuracy and the coding rate increase as they go up to the left in the region between the upper limit curve and the lower limit curve and the quantization accuracy and the coding rate both decrease as they go down to the right.
(Problem of the Second Conventional Example)
The low bit rate coding device for television signals according to the second conventional example determines the normal mode or the fall back mode according to the instantaneous communication condition to control the video data length and the error correcting code length. Accordingly, if the communication condition was good at the time when the normal/fall back mode selection signal was received and then the normal/fall back mode selection signal indicating the normal mode was received, but the communication condition deteriorates at the time of actual transmission, a problem will be encountered. That is to say, in this case, since the normal mode is being selected, coding is made so that the video data length becomes longer and a short (with low error correcting capability) error correcting code is added. However, since the communication condition is bad, and then a transmission error exceeding the error correcting capability will occur to render the decoding on the receiving side impossible.
On the other hand, if the communication condition was bad at the time when the normal/fall back mode selection signal was received and then the normal/fall back mode selection signal indicating the fall back mode was received, but the communication condition becomes good when actually making transmission, such a problem as shown below occurs. That is to say, in this case, since the fall back mode is selected, coding is made so that the video data length becomes shorter and a long (with high error correcting capability) error correcting code is added. Accordingly, the video displayed on the receiving side is deteriorated more than that in the normal mode.
(Problem of the Third Conventional Example)
In the video coding device according to the third conventional example, the priority area and the non-priority area are predetermined and sizes of the areas are constant and unchanged. If movement of the object becomes larger and the amount of generated information increases, or if the communication throughput decreases due to deterioration of conditions of the communication path, then the quantization accuracy must be lowered by coding control to decrease the amount of generated information. At this time, when the amount of generated information extremely increases or the communication throughput decreases with the size of the priority area unchanged, the quantization accuracy not only for the non-priority area but also for the priority area must be largely downed, which will considerably damage the video quality of the priority area.
Accordingly, an object of the present invention is to provide a video coding device which can suppress a delay time so that a moving picture with smooth movement can be displayed on the receiving side even when the amount of generated information rapidly increases due to sudden movement of an object, for example, or when the transmission throughput varies and a video transmission system using the same.
Another object of the present invention is to provide a video coding device capable of most suitable quantization control and selection of error correcting codes corresponding to the communication conditions and a video transmission system using the same.
Still another object of the present invention is to provide a video coding device which can instantaneously determine with simple calculation a quantization step size most suitable in view of visual properties for a video frame after a rapid increase of the amount of generated information (a frame with large motion of the object) and a video transmission system using the same.
A further object of the present invention is to provide a video coding device and a quantization control method which control allocation of a priority area and a non-priority area and the quantization step size according to the amount of generated information and the coding rate (motion of the object) and determine a quantization step size most suitable in view of visual properties for the priority area so that stable video quality can always be kept.
In order to achieve the objects above, the present invention has characteristics as shown below.
According to a first aspect of the present invention, a video coding device for coding and transmitting a moving picture includes:
a video input portion for forming a video frame from a picture signal and outputting the video frame;
a coding portion for coding the video frame formed in the video input portion;
a transmission buffer for storing video data of the video frame coded in the coding portion;
a transmission control portion for transmitting the video data in the transmission buffer at a certain transmission rate;
a temporary buffer disposed between the coding portion and the transmission buffer for temporarily storing the video data of the video frame coded in the coding portion; and
a frame dropping/quantization control portion for determining whether to store into the transmission buffer the video data in the temporary buffer and determining a quantization step size for a video frame coded next;
wherein when the amount of information of the video frame stored in the temporary buffer is larger than a predetermined frame dropping threshold, the frame dropping/quantization control portion performs frame dropping control not to store the video data in the temporary buffer into the transmission buffer and sets a quantization step size with a quantization step size larger than a quantization step size used to code the video frame stored in the temporary buffer and sends the quantization step size to the coding portion, and
the coding portion codes the video frame coded next with the quantization step size sent from the frame dropping/quantization control portion.
As stated above, according to the first aspect, when an object suddenly moves, for example, frame dropping is applied to the video frame with a rapid increase in amount of generated information and the next video frame is coded with a larger quantization step size. This suppresses the amount of generated information and reduces the delay time. Accordingly, a moving picture with smooth movement can be displayed on the receiving side.
According to a second aspect, in the first aspect, when a communication error occurs and the video data can not be normally decoded on a receiving device side, the transmission control portion further makes automatic retransmission and sends error information indicating the occurrence of the retransmission to the frame dropping/quantization control portion, and
the frame dropping/quantization control portion further performs threshold control of lowering the frame dropping threshold for determining whether to store the video data in the temporary buffer into the transmission buffer from the predetermined value when receiving the notice of error information from the transmission control portion, and returning the frame dropping threshold to the predetermined value when not receiving the notice of error information from the transmission control portion in a predetermined constant period of time.
As state above, according to the second aspect, the frame dropping control is made with a decreased frame dropping threshold until the communication condition settles into a good condition. This allows frame dropping control to be performed with the communication condition taken into account.
According to a third aspect, in the first aspect,
the transmission control portion, further, calculates an average throughput in a predetermined constant period of time and sends the average throughput to the frame dropping/quantization control portion, and
the frame dropping/quantization control portion, further,
calculates a maximum amount of the information transmittable in a maximum permissible delay time on basis of a predetermined maximum permissible delay time and the average throughput calculated in the transmission control portion, and
uses the calculated maximum amount of the information as the frame dropping threshold.
As mentioned above, according to the third aspect, the amount of information transmittable in the maximum permissible delay time is calculated from the predetermined maximum permissible delay time and the average throughput and the calculated amount of information is used as a threshold for determining whether the video frame in the temporary buffer is to be dropped or not. This allows frame dropping control corresponding to the communication condition.
According to a fourth aspect, in the third aspect, the frame dropping/quantization control portion, further, calculates an average delay time on the basis of an average amount of the generated information of the video frame transmitted in a constant period of time and the average throughput, and
uses the calculated average delay time as the maximum permissible delay time.
As state above, according to the fourth aspect, an average delay time is calculated from the average amount of the generated information of the video frame transmitted in a constant period of time and the average throughput, and the amount of information transmittable in the average delay time is also calculated, and then the calculated amount of the information is used as the threshold for frame dropping control. This allows frame dropping control corresponding to the moving video characteristics and communication conditions.
According to a fifth aspect, a video coding device for coding and transmitting a moving picture includes:
a video input portion for forming a video frame from a picture signal and outputting the video frame;
a coding portion for coding the video frame formed in the video input portion;
a transmission buffer for storing video data coded in the coding portion;
a first quantization control portion for determining a most suitable quantization step size in view of both reproducibility of motion and video quality;
a second quantization control portion for controlling a quantization step size according to communication conditions; and
a transmission control portion for transmitting the video data in the transmission buffer at a certain transmission rate, and when a communication error occurs and the video data can not be normally decoded on a receiving device side, making automatic retransmission and sending error information indicating the occurrence of the retransmission to the second quantization control portion;
wherein the second quantization control portion,
when receiving the notice of error information from the transmission control portion, sets a quantization step size with a larger quantization step size than a quantization step size corresponding to the quantization step size determined in the first quantization control portion and sends the quantization step size to the coding portion, and
when not receiving the notice of the error information from the transmission control portion in a predetermined constant period of time, sends the quantization step size determined in the first quantization control portion unchanged to the coding portion, and
the coding portion codes a video frame coded next with the quantization step size sent from the second quantization control portion.
As mentioned above, according to the fifth aspect, coding is performed with an increased quantization step size until the communication condition settles into a good condition, so that the delay time can be reduced even when the communication condition is bad and a moving picture with smooth motion can be displayed on the receiving side. Thus, quantization control can be made without affected by fluctuation of the communication condition.
According to a sixth aspect, in the fifth aspect, the video coding device further includes,
a temporary buffer disposed between the coding portion and the transmission buffer for temporarily storing the video data of the video frame coded in the coding portion, and
a frame dropping/quantization control portion for determining whether to store the video data in the temporary buffer into the transmission buffer and determining a quantization step size for a video frame coded next,
wherein the transmission control portion sends the error information indicating occurrence of the retransmission further to the frame dropping/quantization control portion, and
the frame dropping/quantization control portion,
performs threshold control of lowering a frame dropping threshold for determining whether to store the video data in the temporary buffer into the transmission buffer from a predetermined value when receiving the notice of error information from the transmission control portion, and returning the frame dropping threshold to the predetermined value when not receiving the notice of error information from the transmission control portion in a predetermined constant period of time, and
when the amount of information of the video data in the temporary buffer is larger than the frame dropping threshold controlled in the threshold control, performs frame dropping control of not storing the video data in the temporary buffer into the transmission buffer and also sets a quantization step size with a larger quantization step size than a quantization step size used to code the video frame stored in the temporary buffer and sends the quantization step size to the coding portion.
As stated above, according to the sixth aspect, the frame dropping control is made with a lowered frame dropping threshold until the communication condition settles into a good condition. This allows frame dropping control to be performed with the communication condition taken into account.
According to a seventh aspect, in the sixth aspect, the transmission control portion, further, calculates an average throughput in a predetermined constant period of time and sends the average throughput to the frame dropping/quantization control portion, and
the frame dropping/quantization control portion, further,
calculates a maximum amount of the information transmittable in a maximum permissible delay time on basis of a predetermined maximum permissible delay time and the average throughput calculated in the transmission control portion, and
uses the calculated maximum amount of the information as the frame dropping threshold.
As state above, according to the seventh aspect, the amount of information transmittable in the maximum permissible delay time is calculated from the predetermined maximum permissible delay time and the average throughput and the calculated amount of information is used as a threshold for determining whether the video frame in the temporary buffer is to be dropped or not. This allows frame dropping control corresponding to the communication condition.
According to an eighth aspect, in the seventh aspect, the frame dropping/quantization control portion, further, calculates an average delay time on the basis of an average amount of the generated information of the video frame transmitted in a constant period of time and the average throughput, and
uses the calculated average delay time as the maximum permissible delay time.
As mentioned above, according the eighth aspect, an average delay time is calculated from the average amount of generated information of the video frame transmitted in a constant period of time and the average throughput, and the amount of information transmittable in the average delay time is also calculated, and then the calculated amount of the information is used as the threshold for frame dropping control. This allows frame dropping control corresponding to the moving video characteristics and communication condition.
A ninth aspect of the present invention is directed to a video coding device in which operating points is predetermined as a balance between timewise distortion due to frame dropping according to a coding rate, which is a rate of frames transmittable per unit time, and spacial distortion due to a quantization step size when low bit rate compression coding a moving picture and transmitting the coded video data through a communication line with a low bit rate. According to the present invention, the video coding device includes:
a video input portion for forming a video frame from a picture signal and outputting the video frame;
a coding portion for coding the video frame formed in the video input portion;
a transmission buffer for storing the video data coded in the coding portion;
a quantization control portion for determining a quantization step size corresponding to communication conditions; and
a transmission control portion for transmitting the video data in the transmission buffer at a certain transmission rate;
wherein the quantization control portion calculates an average throughput in the constant period of time,
calculates the coding rate on basis of the calculated average throughput and the amount of generated information per one frame resulting from coding in the coding portion,
determines a quantization step size most suitable in view of visual properties corresponding to the calculated coding rate and sends the quantization step size to the coding portion, and
sends the coding portion a notice to code one video frame when the remaining amount of data in the transmission buffer falls below a certain threshold.
As mentioned above, according to the ninth aspect, a coding rate is calculated using an average throughput over a constant period of time and a quantization step size for the next frame is determined using that coding rate. This allows quantization control corresponding to the communication condition.
According to a tenth aspect, in the ninth aspect, the video coding device further includes,
a temporary buffer disposed between the coding portion and the transmission buffer for temporarily storing the video data of the video frame coded in the coding portion, and
a frame dropping/quantization control portion for determining whether to store the video data in the temporary buffer into the transmission buffer and determining a quantization step size for a video frame coded next,
wherein the frame dropping/quantization control portion,
calculates the amount of information transmittable in a maximum permissible delay time on the basis of a predetermined maximum permissible delay time and the average throughput calculated in the quantization control portion, and
performs frame dropping control not to store the video data in the temporary buffer into the transmission buffer if the amount of information of the video frame stored in the temporary buffer is larger than the amount of information transmittable in the maximum permissible delay time and sets a quantization step size with a larger quantization step size than a quantization step size used to code the video frame stored in the temporary buffer and sends the quantization step size to the coding portion.
As mentioned above, according to the tenth aspect, the amount of information transmittable in the maximum permissible delay time is calculated from the predetermined maximum permissible delay time and the average throughput and the calculated amount of information is used as a threshold for determining whether the video frame in the temporary buffer is to be dropped or not. This allows frame dropping control corresponding to the communication condition.
According to an eleventh aspect, in the ninth aspect, the video coding device further includes,
a temporary buffer disposed between the coding portion and the transmission buffer for temporarily storing the video data of the video frame coded in the coding portion, and
a frame dropping/quantization control portion for determining whether to store the video data in the temporary buffer into the transmission buffer and determining a quantization step size for a video frame coded next,
wherein the frame dropping/quantization control portion,
calculates an average delay time on the basis of an average amount of the generated information of the video frame transmitted in a constant period of time and the average throughput calculated in the quantization control portion,
calculates the amount of information transmittable in the average delay time on the basis of the average throughput and the average delay time, and
performs frame dropping control of not storing the video data in the temporary buffer into the transmission buffer if the amount of information of the video frame stored in the temporary buffer is larger than the amount of information transmittable in the average delay time and sets a quantization step size with a larger quantization step size than a quantization step size used to code the video frame stored in the temporary buffer and sends the quantization step size to the coding portion.
As mentioned above, according the eleventh aspect, an average delay time is calculated from the average amount of generated information of the video frame transmitted in a constant period of time and the average throughput, and the amount of information transmittable in the average delay time is also calculated, and then the calculated amount of the information is used as the threshold for frame dropping control. This allows frame dropping control corresponding to the moving video characteristics and the communication condition.
According to a twelfth aspect, in the ninth aspects, when the quantization control portion calculates the average throughput, the time over which the average is calculated is variable according to whether a burst error occurs or not.
As stated above, according to the twelfth aspect, decreasing the time over which the average is obtained allows quantization control and frame dropping control quickly responsive to a change in throughput, and in the condition where burst type errors occur, setting the time over which the average is obtained longer than the cycle of the occurrence of the burst errors allows quantization control and frame dropping control to keep stable even with burst errors.
According to a thirteenth aspect, in the ninth aspect, the predetermined constant period of time for calculating the average throughput is not less than 300 msec.
As mentioned above, according to the thirteenth aspect, the time for obtaining the average of 300 msec or longer permits quantization control and frame dropping control which do not go unstable even with occurrence of the burst error.
According to a fourteenth aspect, a video coding device for coding and transmitting a moving picture includes:
a video input portion for forming a video frame from a picture signal and outputting the video frame;
a coding portion for coding the video frame formed in the video input portion;
a transmission buffer for storing the video data of the video frame coded in the coding portion;
a quantization control portion for determining a quantization step size used in coding;
a transmission control portion for transmitting the video data in the transmission buffer at a certain transmission rate;
a motion threshold storing portion for storing a motion threshold for determining whether motion of an object is large or small; and
a maximum permissible delay time storing portion for storing a maximum permissible delay time which is a limit of a transmission delay which does not cause visually unnatural impression;
wherein when the amount of generated information per one frame resulting from coding in the coding portion is larger than the motion threshold, the quantization control portion,
obtains a quantization ratio which is a ratio of a quantization step size used to code the video frame exceeding the motion threshold and a quantization step size for which the amount of generated information is to be predicted,
predicts the amount of information generated when the quantization step size is changed from the amount of generated information of the video frame exceeding the motion threshold and the quantization ratio, and
determines a quantization step size transmittable in the maximum permissible delay time from the predicted amount of generated information, the transmission rate and the maximum permissible delay time and sends the quantization step size to the coding portion, and
the coding portion codes the video frame with the quantization step size sent from the quantization control portion.
As stated above, according to the fourteenth aspect, since the ratio of the quantization step size is used to predict the amount of generated information utilizing the characteristics that the amount of generated information of the video frame is approximately in inverse proportion to the quantization step size, when the motion of the object is large, the amount of generated information can be predicted with simple calculation and the quantization step size transmittable in the maximum permissible delay time can be determined instantaneously after the large motion of the object.
According to a fifteenth aspect, in the fourteenth aspect, the transmission control portion, further, calculates an average throughput in a predetermined constant period of time and sends the average throughput to the quantization control portion, and
if the amount of generated information per one frame resulting from coding in the coding portion is larger than the motion threshold, the quantization control portion,
obtains a quantization ratio which is a ratio of a quantization step size used to code the video frame exceeding the motion threshold and a quantization step size for which the amount of generated information is to be predicted,
predicts the amount of information generated when the quantization step size is changed from the amount of generated information of the video frame exceeding the motion threshold and the quantization ratio, and
determines a quantization step size transmittable in the maximum permissible delay time from the predicted amount of the generated information, the transmission rate, the average throughput and the maximum permissible delay time and sends the quantization step size to the coding portion.
As mentioned above, according to the fifteenth aspect, since the average throughput is used to determine the quantization step size in the case in which the motion of the object is large, a quantization step size corresponding to the change in throughput can be determined.
According to a sixteenth aspect, in the fifteenth aspect, when the transmission control portion calculates the average throughput, the time over which the average is obtained is variable according to whether a burst error occurs or not.
As state above, according to the sixteenth aspect, decreasing the time for obtaining the average allows determination of the quantization step size when the motion of the object is large quickly responsive to a change in throughput, and in the condition where burst type errors occur, setting the time for obtaining the average longer than the cycle of the occurrence of the burst errors allows determination of the quantization step size to keep stable even with burst errors.
According to a seventeenth aspect, in the fifteenth aspect, the predetermined constant period of time for calculating the average throughput is not less than 300 msec.
As mentioned above, according to the seventeenth aspect, setting the time for obtaining the average to 300 msec or longer permits determination of the quantization step size when the motion of the object is large to keep stable even at the time of occurrence of the burst error.
According to an eighteenth aspect, a video coding device for coding and transmitting a moving picture includes:
a video input portion for forming a video frame from a picture signal and outputting the video frame;
a coding portion for coding the video frame formed in the video input portion;
a transmission buffer for storing video data of the video frame coded in the coding portion;
a quantization control portion for determining a quantization step size used in coding;
a transmission control portion for transmitting the video data in the transmission buffer at a certain transmission rate;
a motion threshold storing portion for storing a motion threshold for determining whether motion of an object is large or small; and
an ideal curve storing portion for storing an ideal curve with predetermined operating points most suitable in view of visual properties as a balance between timewise distortion due to frame dropping according to a coding rate which is a rate of frames transmittable per unit time and spacial distortion due to a quantization step size;
wherein if the amount of generated information per one frame resulting from coding in the coding portion is larger than the motion threshold, the quantization control portion,
obtains a quantization ratio which is a ratio of a quantization step size used to code the video frame exceeding the motion threshold and a quantization step size for which the amount of generated information is to be predicted,
predicts the amount of information generated when the quantization step size is changed from the amount of generated information of the video frame exceeding the motion threshold and the quantization ratio,
calculates the coding rate from the predicted amount of the generated information and the transmission rate, and
determines a quantization step size most suitable in view of the visual properties from the calculated coding rate and the ideal curve and sends the quantization step size to the coding portion, and
the coding portion codes the video frame with the quantization step size sent from the quantization control portion.
As stated above, according to the eighteenth aspect, like the fourteenth aspect, the amount of generated information can be predicted with simple calculation and a quantization step size most suitable in view of visual properties can be determined from the ideal curve and the predicted amount of the generated information instantaneously after the large motion of the object.
According to a nineteenth aspect, in the eighteenth aspect, the transmission control portion, further, calculates an average throughput in a predetermined constant period of time and sends the average throughput to the quantization control portion, and
when the amount of generated information per one frame resulting from coding in the coding portion is larger than the motion threshold, the quantization control portion,
obtains a quantization ratio which is a ratio of a quantization step size used to code the video frame exceeding the motion threshold and a quantization step size the amount of generated information for which is to be predicted,
predicts the amount of information generated when the quantization step size is changed from the amount of generated information of the video frame exceeding the motion threshold and the quantization ratio,
calculates the coding rate from the predicted amount of the generated information, the transmission rate and the average throughput, and
determines a quantization step size most suitable in view of visual properties from the calculated coding rate and the ideal curve and sends the quantization step size to the coding portion.
As mentioned above, according to the nineteenth aspect, since the average throughput and the ideal curve are used to determine the quantization step size when the motion of the object is large, the visually most suitable quantization step size can be determined taking the change of the throughput into account.
According to a twentieth aspect, in the nineteenth aspect, when the transmission control portion calculates the average throughput, the time over which the average is obtained is variable according to whether a burst error occurs or not.
As state above, according to the twentieth aspect, decreasing the time for obtaining the average allows the determination of the quantization step size when the motion of the object is large to quickly respond to the change in throughput, and in the condition where burst type errors occur, setting the time over which the average is obtained longer than the cycle of the occurrence of the burst errors allows the determination of the quantization step size to keep stable even with burst errors.
According to a twenty-first aspect, in the nineteenth aspect, the predetermined constant period of time for calculating the average throughput is not less than 300 msec.
As mentioned above, according to the twenty-first aspect, the time for obtaining the average of 300 msec or longer permits determination of quantization step size for the case in which the motion of the object is large which do not go unstable even with occurrence of a burst error.
A twenty-second aspect of the present invention is directed to a video transmission system in which a video coding device for coding and transmitting a moving picture and a video receiving device for applying certain processing to received video data are connected so that they can communicate,
wherein the video receiving device calculates an error rate of the received video data and if the calculated error rate exceeds a certain threshold, transmits an error signal to the video coding device, and
the video coding device includes,
a video input portion for forming a video frame from a picture signal and outputting the video frame,
a coding portion for coding the video frame formed in the video input portion,
a transmission buffer for storing the video data coded in the coding portion,
a transmission control portion for transmitting the video data in the transmission buffer at a certain transmission rate,
a first quantization control portion for determining a most suitable quantization step size in view of both reproducibility of motion and video quality,
an error correction coding portion for adding error correcting codes with different error correcting capabilities to the video data coded in the coding portion, and
a second quantization control portion for determining a quantization step size corresponding to communication conditions and controlling the error correction coding portion according to the error signal,
wherein the second quantization control portion,
when receiving the error signal from the video receiving device, sets a quantization step size with a larger quantization step size than a quantization step size corresponding to the quantization step size determined in the first quantization control portion and sends the quantization step size to the coding portion and outputs a selection signal instructing the error correction coding portion to add an error correcting code with a high error correcting capability, and
if not receiving the notice of the error signal in a predetermined constant period of time, sends the quantization step size determined in the first quantization control portion unchanged to the coding portion and outputs a selection signal instructing the error correction coding portion to add an error correcting code with a low error correcting capability,
the coding portion codes the video frame with the quantization step size sent from the second quantization control portion, and
the error correction coding portion adds the error correcting code indicated by the selection signal to the video data coded in the coding portion.
As described above, according to the twenty-second aspect, coding is made with an increased quantization step size until the communication condition settles into a good condition and the coding is made with an error correcting code with a high error correcting capability. This does not cause the disadvantage of coding with a quantization step size having a large quantization step size and an error correcting code with a high error correcting capability in spite of a good communication condition, or conversely, the disadvantage of coding with a quantization step size having a small quantization step size and an error correcting code with a low error correcting capability in spite of a bad communication condition. Accordingly, coding control can be made in correspondence with the communication condition.
According to a twenty-third aspect, in the twenty-second aspect, the video receiving device calculates an average of the error rate calculated in a predetermined constant period of time and if the calculated average error rate exceeds a certain threshold, sends the error signal to the video coding device.
As state above, according to the twenty-third aspect, the video receiving device calculates an average of the error rate, and when the average error rate exceeds the predetermined threshold, it transmits an error signal to the video coding device so that the coding control unaffected by temporary good/bad communication condition can be made in the video coding device.
According to a twenty-fourth aspect, in the twenty second or twenty-third aspect, the video coding device further includes,
a temporary buffer disposed between the coding portion and the transmission buffer for temporarily storing the video data of the video frame coded in the coding portion, and
a frame dropping/quantization control portion for determining whether to store the video data in the temporary buffer into the transmission buffer and determining a quantization step size for a video frame coded next,
wherein the frame dropping/quantization control portion,
performs threshold control of lowering a frame dropping threshold for determining whether to store the video data in the temporary buffer into the transmission buffer from a predetermined value when receiving the error signal from the video receiving device, and when not receiving notice of the error signal in a predetermined constant period of time, returning the frame dropping threshold to the predetermined value, and
if the amount of information of the video data in the temporary buffer is larger than the frame dropping threshold controlled in the threshold control, performs frame dropping control not to store the video data in the temporary buffer into the transmission buffer, and also sets a quantization step size with a larger quantization step size than a quantization step size used to code the video frame stored in the temporary buffer and sends the quantization step size to the coding portion.
As stated above, according to the twenty-fourth aspect, since frame dropping control is performed with a decreased frame dropping threshold until the communication condition settles into a good condition, the frame dropping control can be made taking the communication condition into consideration.
A twenty-fifth aspect of the present invention is directed to a video transmission system in which a video coding device for coding and transmitting a moving picture and a video receiving device for applying certain processing to received video data are connected so that they can communicate,
wherein the video receiving device calculates an error rate of the received video data and transmits the error rate to the video coding device, and
the video coding device includes,
a video input portion for forming a video frame from a picture signal and outputting the video frame,
a coding portion for coding the video frame formed in the video input portion,
a transmission buffer for storing the video data coded in the coding portion,
a transmission control portion for transmitting the video data in the transmission buffer at a certain transmission rate,
a first quantization control portion for determining a most suitable quantization step size in view of both reproducibility of motion and video quality,
an error correction coding portion for adding error correcting codes with different error correcting capabilities to the video data coded in the coding portion, and
a second quantization control portion for determining a quantization step size corresponding to communication conditions and controlling the error correction coding portion according to the error rate,
wherein the second quantization control portion,
if the error rate received from the video receiving device exceeds a predetermined threshold, sets a quantization step size with a larger quantization step size than a quantization step size corresponding to the quantization step size determined in the first quantization control portion and sends the quantization step size to the coding portion, and outputs a selection signal instructing the error correction coding portion to add the error correcting code with a high error correcting capability, and
if the error rate does not exceed the threshold in a predetermined constant period of time, sends the quantization step size determined in the first quantization control portion unchanged to the coding portion and outputs a selection signal instructing the error correction coding portion to add the error correcting code with a low error correcting capability,
the coding portion codes the video frame with the quantization step size sent from the second quantization control portion, and
the error correction coding portion adds the error correcting code indicated by the selection signal to the video data coded in the coding portion.
As mentioned above, according to the twenty-fifth aspect, it is determined whether the communication condition is good or bad according to the error rate from the video receiving device and coding is performed with an increased quantization step size until the communication condition settles into a good condition, and the coding is made with an error correcting code with higher error correcting capability, which permits coding control to be done corresponding to the communication condition.
According to a twenty-sixth aspect, in the twenty-fifth aspect, the second quantization control portion,
calculates an average of the error rate received from the video receiving device in a predetermined constant period of time,
if the calculated average error rate exceeds a predetermined threshold, sets a quantization step size with a larger quantization step size than a quantization step size corresponding to the quantization step size determined in the first quantization control portion and sends the quantization step size to the coding portion and outputs a selection signal instructing the error correction coding portion to add the error correcting code with a high correcting capability, and
if the average error rate does not exceed the threshold in a predetermined constant period of time, sends the quantization step size determined in the first quantization control portion unchanged to the coding portion and outputs a selection signal instructing the error correction coding portion to add the error correcting code with a low error correcting capability.
As stated above, according to the twenty-sixth aspect, an average of the error rate from the video receiving device is calculated in the video coding device and it is determined whether the communication condition is good or bad on the basis of the calculated average error rate. The coding is performed with an increased quantization step size until the communication condition settles into good condition, and the coding is made with an error correcting code with higher error correcting capability, which permits coding control to be done corresponding to the communication condition.
According to a twenty-seventh aspect, in the twenty-sixth aspect, the video coding device further includes,
a temporary buffer disposed between the coding portion and the transmission buffer for temporarily storing the video data of the video frame coded in the coding portion, and
a frame dropping/quantization control portion for determining whether to store the video data in the temporary buffer into the transmission buffer and determining a quantization step size for a video frame coded next,
wherein the frame dropping/quantization control portion,
performs threshold control of lowering a frame dropping threshold for determining whether to store the video data in the temporary buffer into the transmission buffer from a predetermined value when the average error rate calculated in the second quantization control portion exceeds a predetermined threshold, and returning the frame dropping threshold to the predetermined value when the average error rate does not exceed the threshold in a predetermined constant period of time, and
performs frame dropping control of not storing the video data in the temporary buffer into the transmission buffer if the amount of information of the video data in the temporary buffer is larger than the frame dropping threshold controlled in the threshold control and also sets a quantization step size with a larger quantization step size than a quantization step size used to code the video frame stored in the temporary buffer and sends the quantization step size to the coding portion.
As mentioned above, according to the twenty-seventh aspect, an average of the error rate from the video receiving device is calculated and it is determined whether the communication condition is good or bad on the basis of the calculated average error rate. Then coding control is performed with a decreased frame dropping threshold until the communication condition settles into good condition, which permits coding control to be done with the communication condition taken into account.
According to a twenty-eighth aspect, in the twenty-fifth aspect, the video coding device further includes,
a temporary buffer disposed between the coding portion and the transmission buffer for temporarily storing the video data of the video frame coded in the coding portion, and
a frame dropping/quantization control portion for determining whether to store the video data in the temporary buffer into the transmission buffer and determining a quantization step size for a video frame coded next,
wherein the frame dropping/quantization control portion,
performs threshold control of lowering a frame dropping threshold for determining whether to store the video data in the temporary buffer into the transmission buffer from a predetermined value when the error rate received from the video receiving device exceeds a predetermined threshold, and returning the frame dropping threshold to the predetermined value when the error rate does not exceed the threshold in a predetermined constant period of time, and
performs frame dropping control not to store the video data in the temporary buffer into the transmission buffer if the amount of information of the video data in the temporary buffer is larger than the frame dropping threshold controlled in the threshold control and also sets a quantization step size with a larger quantization step size than a quantization step size used to code the video frame stored in the temporary buffer and sends the quantization step size to the coding portion.
As mentioned above, according to the twenty-eighth aspect, the condition of the communication is determined in the video coding device on the basis of the error rate from the video receiving device, and frame dropping control is performed with a decreased frame dropping threshold until the communication condition settles into a good condition, which permits coding control taking the communication condition into consideration.
A twenty-ninth aspect of the present invention relates to a video transmission system in which a video coding device for coding and transmitting a moving picture and a video receiving device for applying certain processing to received video data are connected so that they can communicate,
wherein the video coding device obtains an error rate of the received video data, calculates an average of the error rate obtained in a predetermined constant period of time, and transmits the calculated average error rate to the video coding device,
the video coding device includes,
a video input portion for forming a video frame from a picture signal and outputting the video frame,
a coding portion for coding the video frame formed in the video input portion,
a transmission buffer for storing the video data coded in the coding portion,
a transmission control portion for transmitting the video data in the transmission buffer at a certain transmission rate,
a first quantization control portion for determining a most suitable quantization step size in view of both reproducibility of motion and video quality,
an error correction coding portion for adding error correcting codes with different error correcting capabilities to the video data coded in the coding portion, and
a second quantization control portion for determining a quantization step size corresponding to communication conditions and controlling the error correction coding portion according to the average error rate,
wherein the second quantization control portion,
when the average error rate received from the video receiving device exceeds a predetermined threshold, sets a quantization step size with a larger quantization step size than a quantization step size corresponding to the quantization step size determined in the first quantization control portion and sends the quantization step size to the coding portion and outputs a selection signal instructing the error correction coding portion to add the error correcting code with a high error correcting capability, and
when the average error rate does not exceed the threshold in a predetermined constant period of time, sends the quantization step size determined in the first quantization control portion unchanged to the coding portion and outputs a selection signal instructing the error correction coding portion to add the error correcting code with a low error correcting capability,
the coding portion codes the video frame with the quantization step size sent from the second quantization control portion, and
the error correction coding portion adds the error correcting code indicated by the selection signal to the video data coded in the coding portion.
As state above, according to the twenty-ninth aspect, good/bad of the communication condition is determined on the basis of the average error rate from the video receiving device, and coding is performed with an increased quantization step size until the communication condition settles into good condition, and the coding is performed with an error correcting code with high error correcting capability, which permits coding control unaffected by temporary communication condition.
According to a thirtieth aspect, in the twenty-ninth aspect, the video coding device further includes,
a temporary buffer disposed between the coding portion and the transmission buffer for temporarily storing the video data of the video frame coded in the coding portion, and
a frame dropping/quantization control portion for determining whether to store the video data in the temporary buffer into the transmission buffer and determining a quantization step size for a video frame coded next,
wherein the frame dropping/quantization control portion,
performs threshold control of lowering a frame dropping threshold for determining whether to store the video data in the temporary buffer into the transmission buffer from a predetermined value when the average error rate received from the video receiving device exceeds a predetermined threshold, and returning the frame dropping threshold to the predetermined value when the average error rate does not exceed the threshold in a predetermined constant period of time, and
performs frame dropping control of not storing the video data in the temporary buffer into the transmission buffer if the amount of information of the video data in the temporary buffer is larger than the frame dropping threshold controlled in the threshold control and also sets a quantization step size with a larger quantization step size than a quantization step size used to code the video frame stored in the temporary buffer and sends the quantization step size to the coding portion.
As mentioned above, according to the thirtieth aspect, the video coding device determines whether the communication condition is good or deteriorated on the basis of the average error rate from the video receiving device, and performs frame dropping control with a decreased frame dropping threshold until the communication condition settles into a good condition, which permits frame dropping control unaffected by temporary communication condition.
A thirty-first aspect of the present invention is intended for a video transmission system in which a video coding device for coding and transmitting a moving picture and a video receiving device for applying certain processing to received video data are communicatably connected,
wherein the video receiving device calculates an error rate of the received video data and transmits an error signal to the video coding device if the calculated error rate exceeds a certain threshold, and
the video coding device includes,
a video input portion for forming a video frame from a picture signal and outputting the video frame,
a coding portion for coding the video frame formed in the video input portion,
a transmission buffer for storing the video data coded in the coding portion,
a transmission control portion for transmitting the video data in the transmission buffer at a certain transmission rate,
a temporary buffer disposed between the coding portion and the transmission buffer for temporarily storing the video data of the video frame coded in the coding portion,
a frame dropping/quantization control portion for determining whether to store into the transmission buffer the video data in the temporary buffer and determining a quantization step size for a video frame coded next, and
an error correction coding portion for adding error correcting codes with different error correcting capabilities to the video data in the temporary buffer and storing the data in the transmission buffer,
wherein the frame dropping/quantization control portion,
performs threshold control of lowering a frame dropping threshold for determining whether to store the video data in the temporary buffer into the transmission buffer from a predetermined value when receiving the error signal, and outputs a selection signal instructing the error correction coding portion to add an error correcting code with a high error correcting capability,
performs threshold control of returning the frame dropping threshold to the predetermined value if not receiving notice of the error signal in a predetermined constant period of time, and outputs a selection signal instructing the error correction coding portion to add an error correcting code with a low error correcting capability, and
performs frame dropping control not to store into the transmission buffer the video data in the temporary buffer if the amount of information of the video data in the temporary buffer is larger than the frame dropping threshold controlled in the threshold control and also sets a quantization step size with a larger quantization step size than a quantization step size used to code the video frame stored in the temporary buffer and sends the quantization step size to the coding portion, and
the error correction coding portion adds the error correcting code indicated by the selection signal to the video data coded in the coding portion.
As state above, according to the thirty-first aspect, since frame dropping control is performed with a decreased frame dropping threshold until the communication condition settles into a good condition, the frame dropping control can be made taking the communication condition into consideration.
According to a thirty-second aspect, in the thirty-first aspect, the video receiving device calculates an average of the error rate calculated in a predetermined constant period of time and if the calculated average error rate exceeds a certain threshold, sends the error signal to the video coding device.
As mentioned above, according to the thirty-second aspect, the video receiving device calculates an average of the error rate and if the average error rate exceeds a predetermined threshold, it sends an error signal to the video coding device so that the video coding device can perform frame dropping control unaffected by temporary good/bad communication condition.
A thirty-third aspect relates to a video transmission system in which a video coding device for coding and transmitting a moving picture and a video receiving device for applying certain processing to received video data are communicatably connected,
wherein the video receiving device calculates an error rate of the received video data and transmits the error rate to the video coding device, and
the video coding device includes,
a video input portion for forming a video frame from a picture signal and outputting the video frame,
a coding portion for coding the video frame formed in the video input portion,
a transmission buffer for storing the video data coded in the coding portion,
a transmission control portion for transmitting the video data in the transmission buffer at a certain transmission rate,
a temporary buffer disposed between the coding portion and the transmission buffer for temporarily storing the video data of the video frame coded in the coding portion,
a frame dropping/quantization control portion for determining whether to store the video data in the temporary buffer into the transmission buffer and determining a quantization step size for a video frame coded next, and
an error correction coding portion for adding error correcting codes with different error correcting capabilities to the video data in the temporary buffer and storing the data in the transmission buffer,
wherein the frame dropping/quantization control portion,
performs threshold control of lowering a frame dropping threshold for determining whether to store into the transmission buffer the video data in the temporary buffer from a predetermined value when the error rate exceeds a predetermined threshold, and outputs a selection signal instructing the error correction coding portion to add the error correcting code with a high error correcting capability,
performs threshold control of returning the frame dropping threshold to the predetermined value when the error rate does not exceed the predetermined threshold in a predetermined constant period of time, and outputs a selection signal instructing the error correction coding portion to add the error correcting code with a low error correcting capability, and
performs frame dropping control of not storing the video data in the temporary buffer into the transmission buffer if the amount of information of the video data in the temporary buffer is larger than the frame dropping threshold controlled in the threshold control and also sets a quantization step size with a larger quantization step size than a quantization step size used to code the video frame stored in the temporary buffer and sends the quantization step size to the coding portion, and
the error correction coding portion adds the error correcting code indicated by the selection signal to the video data coded in the coding portion.
As state above, according to the thirty-third aspect, the video coding device determines whether the communication condition is good/deteriorated on the basis of the error rate from the video receiving device, and frame dropping control is performed with a decreased frame dropping threshold until the communication condition settles into good condition, which permits frame dropping control taking the communication condition into account.
According to a thirty-fourth aspect, in the thirty-third aspect, the video coding device further includes an average error rate calculating portion for calculating an average error rate which is an average of the error rate received from the video receiving device in a predetermined constant period of time,
wherein the frame dropping/quantization control portion,
performs threshold control of lowering a frame dropping threshold for determining whether to store the video data in the temporary buffer into the transmission buffer from a predetermined value when the average error rate exceeds a predetermined threshold, and outputs a selection signal instructing the error correction coding portion to add an error correcting code with a high error correcting capability,
performs threshold control of returning the frame dropping threshold to the predetermined value if the average error rate does not exceed the predetermined threshold in a predetermined constant period of time, and outputs a selection signal instructing the error correction coding portion to add an error correcting code with a low error correcting capability, and
performs frame dropping control of not storing the video data in the temporary buffer into the transmission buffer if the amount of information of the video data in the temporary buffer is larger than the frame dropping threshold controlled in the threshold control and also sets a quantization step size with a larger quantization step size than a quantization step size used to code the video frame stored in the temporary buffer and sends the quantization step size to the coding portion, and
the error correction coding portion adds the error correcting code indicated by the selection signal to the video data coded in the coding portion.
As stated above, according to the thirty-fourth aspect, an average of the error rate from the video receiving device is calculated and good/bad of the communication condition is determined on the basis of the calculated average error rate. Then frame dropping control is performed with a decreased frame dropping threshold until the communication condition settles into a good condition, which permits frame dropping control to be done taking the communication condition into consideration.
A thirty-fifth aspect is directed to a video transmission system in which a video coding device for coding and transmitting a moving picture and a video receiving device for applying certain processing to received video data are communicably connected,
wherein the video receiving device obtains an error rate of the received video data, calculates an average of the error rate obtained in a predetermined constant period of time and transmits the calculated average error rate to the video coding device, and
the video coding device includes,
a video input portion for forming a video frame from a picture signal and outputting the video frame,
a coding portion for coding the video frame formed in the video input portion,
a transmission buffer for storing the video data coded in the coding portion,
a transmission control portion for transmitting the video data in the transmission buffer at a certain transmission rate,
a temporary buffer disposed between the coding portion and the transmission buffer for temporarily storing the video data of the video frame coded in the coding portion,
a frame dropping/quantization control portion for determining whether to store the video data in the temporary buffer into the transmission buffer and determining a quantization step size for a video frame coded next, and
an error correction coding portion for adding error correcting codes with different error correcting capabilities to the video data in the temporary buffer and storing the data in the transmission buffer,
wherein the frame dropping/quantization control portion,
when the average error rate exceeds a predetermined threshold, performs threshold control of lowering a frame dropping threshold for determining whether to store the video data in the temporary buffer into the transmission buffer from a predetermined value, and outputs a selection signal instructing the error correction coding portion to add an error correcting code with a high error correcting capability,
when the average error rate does not exceed the predetermined threshold in a predetermined constant period of time, performs threshold control of returning the frame dropping threshold to the predetermined value, and outputs a selection signal instructing the error correction coding portion to add an error correcting code with a low error correcting capability, and
when the amount of information of the video data in the temporary buffer is larger than the frame dropping threshold controlled in the threshold control, performs frame dropping control of not storing the video data in the temporary buffer into the transmission buffer, and also sets a quantization step size with a larger quantization step size than a quantization step size used to code the video frame stored in the temporary buffer and sends the quantization step size to the coding portion, and
the error correction coding portion adds the error correcting code indicated by the selection signal to the video data coded in the coding portion.
As described above, according to the thirty-fifth aspect, the video coding device determines whether the communication condition is good or bad on the basis of the average error rate from the video receiving device, and frame dropping control is performed with a decreased frame dropping threshold until the communication condition settles into a good condition, which permits frame dropping control unaffected by temporary communication condition.
A thirty-sixth aspect is intended for a video transmission system in which a video coding device for coding and transmitting a moving picture and a video receiving device for applying certain processing to received video data are connected such that they can communicate,
wherein the video receiving device counts the number of bits of data correctly received in a predetermined constant period of time and transmits the counted number of bits to the video coding device, and
the video coding device includes,
a video input portion for forming a video frame from a picture signal and outputting the video frame,
a coding portion for coding the video frame formed in the video input portion,
a transmission buffer for storing the video data coded in the coding portion,
a quantization control portion for determining a quantization step size corresponding to communication conditions, and
a transmission control portion for transmitting the video data in the transmission buffer at a certain transmission rate and outputting the bit number received from the video receiving device to the quantization control portion,
wherein the quantization control portion,
calculates an average throughput in a constant period of time on the basis of the bit number received from the transmission control portion,
calculates a coding rate on the basis of the calculated average throughput and the amount of generated information per one frame resulting from coding in the coding portion, and
determines a quantization step size most suitable in view of visual properties corresponding to the calculated coding rate and sends the quantization step size to the coding portion.
As state above, according to the thirty-sixth aspect, as the video coding device side can know the number of bits correctly transmitted to the video receiving device in a constant period of time, quantization control most suitable for the throughput between the two devices can be made even with a plurality of transmission lines with different throughputs on the communication line between the two devices.
According to a thirty-seventh aspect, in the thirty-sixth aspect, when the quantization control portion calculates the average throughput, the time over which the average is obtained is variable according to the variation of the bit number received from the transmission control means.
As stated above, according to the thirty-seventh aspect, if the time for obtaining the average is shortened, the quantization control can quickly respond to a change in throughput. In the condition where burst type errors occur, if the time for obtaining the average is set longer than the cycle of the occurrence of the burst errors, the quantization control can keep stable even with the burst errors.
According to a thirty-eighth aspect, in the thirty-sixth aspect, the predetermined constant period of time for calculating the average throughput is not less than 300 msec.
As mentioned above, according to the thirty-eighth aspect, setting the time for obtaining the average to 300 msec or longer permits quantization control which do not go unstable even with occurrence of a burst error.
A thirty-ninth aspect is directed to a video transmission system in which a video coding device for coding and transmitting a moving picture and a video receiving device for applying certain processing to received video data are connected so that they can communicate,
wherein the video receiving device counts the number of bits of data correctly received in a predetermined constant period of time, calculates an average throughput from the counted bit number and transmits the average throughput to the video coding device, and
the video coding device includes,
a video input portion for forming a video frame from a picture signal and outputting the video frame,
a coding portion for coding the video frame formed in the video input portion,
a transmission buffer for storing the video data coded in the coding portion,
a quantization control portion for determining a quantization step size corresponding to communication conditions, and
a transmission control portion for transmitting the video data in the transmission buffer at a certain transmission rate and outputting the average throughput received from the video receiving device to the quantization control portion,
wherein the quantization control portion,
calculates a coding rate on the basis of the average throughput received from the transmission control portion and the amount of generated information per one frame resulting from coding in the coding portion, and
determines a quantization step size most suitable in view of visual properties corresponding to the calculated coding rate and sends the quantization step size to the coding portion.
As stated above, according to the thirty-ninth aspect, the video receiving device calculates the average throughput from the number of bits correctly received in a constant period of time and transmits it to the video coding device. Therefore, even with a plurality of transmission lines having different throughputs on the communication line between the two devices, the quantization control most suitable for the throughput between the two devices can be made.
A fortieth aspect is directed to a video transmission system in which a video coding device coding and transmitting a moving picture and a video receiving device applying certain processing to received video data are communicatably connected through one or more relay stations,
wherein when detecting an error of the received data, the video receiving device makes automatic retransmission between adjacent one of the relay stations and itself and transmits retransmission occurrence information indicating the occurrence of the retransmission to the video coding device, and
the video coding device includes,
a video input portion for forming a video frame from a picture signal and outputting the video frame,
a coding portion for coding the video frame formed in the video input portion,
a transmission buffer for storing the video data coded in the coding portion,
a first quantization control portion for determining a quantization step size most suitable in view of both reproducibility of motion and video quality,
a second quantization control portion for controlling a quantization step size according to communication conditions, and
a transmission control portion for transmitting the video data in the transmission buffer at a certain transmission rate, and when a communication error occurs between adjacent one of the relay stations and the video coding device, making automatic retransmission between the adjacent relay station and the video coding device, and when the transmission control portion itself makes retransmission, or when it receives the retransmission occurrence information from the video receiving device, sending error information indicating the occurrence of the retransmission to the second quantization control portion,
wherein the second quantization control portion,
when receiving the notice of the error information from the transmission control portion, sets a quantization step size with a larger quantization step size than a quantization step size corresponding to the quantization step size determined in the first quantization control portion and sends the quantization step size to the coding portion, and
when not receiving the notice of the error information from the transmission control portion over a predetermined constant period of time, sends the quantization step size determined in the first quantization control portion unchanged to the coding portion, and
the coding portion codes a video frame coded next with the quantization step size sent from the second quantization control portion.
As stated above, according to the fortieth aspect, when there are a plurality of transmission lines between two devices and the video coding device and the video receiving device do not directly make retransmission control with each other but they make retransmission control with adjacent ones of relay stations, the delay time can be reduced so that a moving picture with smooth movement can be displayed on the receiving side even if retransmission is made on either side and the communication condition deteriorates.
According to a forty-first aspect, in the fortieth aspect, the video coding device further includes,
a temporary buffer disposed between the coding portion and the transmission buffer for temporarily storing the video data of the video frame coded in the coding portion, and
a frame dropping/quantization control portion for determining whether to store into the transmission buffer the video data in the temporary buffer and determining a quantization step size for a video frame coded next,
wherein the transmission control portion sends the error information indicating the occurrence of the retransmission further to the frame dropping/quantization control portion, and
the frame dropping/quantization control portion,
performs threshold control of lowering a frame dropping threshold for determining whether to store the video data in the temporary buffer into the transmission buffer from a predetermined value when receiving notice of the error information from the transmission control portion, and when not receiving the notice of the error information from the transmission control portion in a predetermined constant period of time, returning the frame dropping threshold to the predetermined value, and
performs frame dropping control of not storing the video data in the temporary buffer into the transmission buffer if the amount of information of the video data in the temporary buffer is larger than the frame dropping threshold controlled in the threshold control and also sets a quantization step size with a larger quantization step size than a quantization step size used to code the video frame stored in the temporary buffer and sends the quantization step size to the coding portion.
As explained above, according to the forty-first aspect, since frame dropping control is made with a decreased frame dropping threshold until the communication condition settles into a good condition, frame dropping control can be made taking the communication condition into account.
A forty-second aspect of the present invention is directed to a video transmission system in which a video coding device for coding and transmitting a moving picture and a video receiving device for applying certain processing to received video data are connected such that they can make communication,
wherein the video receiving device counts the number of bits of data correctly received in a predetermined constant period of time and transmits the counted number of bits to the video coding device, and
the video coding device includes,
a video input portion for forming a video frame from a picture signal and outputting the video frame,
a coding portion for coding the video frame formed in the video input portion,
a transmission buffer for storing the video data of the video frame coded in the coding portion,
a temporary buffer disposed between the coding portion and the transmission buffer for temporarily storing the video data of the video frame coded in the coding portion,
a frame dropping/quantization control portion for determining whether to store the video data in the temporary buffer into the transmission buffer and determining a quantization step size for a video frame coded next, and
a transmission control portion for transmitting the video data in the transmission buffer at a certain transmission rate and calculating an average throughput from the bit number received from the video receiving device,
wherein the frame dropping/quantization control portion,
calculates the amount of information transmittable in a maximum permissible delay time on the basis of a predetermined maximum permissible delay time and the average throughput, and
performs frame dropping control of not storing into the transmission buffer the video data in the temporary buffer if the amount of information of the video frame stored in the temporary buffer is larger than the amount of information transmittable in the maximum permissible delay time and also sets a quantization step size with a larger quantization step size than a quantization step size used to code the video frame stored in the temporary buffer and sends the quantization step size to the coding portion.
As mentioned above, according to the forty-second aspect, as the video coding device side can know the number of bits correctly transmitted to the video receiving device in a constant period of time, most suitable frame dropping control taking the throughput between the two devices into account can be made even when a plurality of transmission lines with different throughputs exist on the communication line between the two devices.
According to a forty-third aspect, in the forty-second aspect, the frame dropping/quantization control portion, further,
calculates an average delay time on the basis of an average amount of the generated information of the video frame transmitted in a constant period of time and the average throughput, and
uses the calculated average delay time as the maximum permissible delay time.
As mentioned above, according to the forty-third aspect, an average delay time is calculated from the average amount of the generated information of the video frame transmitted in a constant period of time and the average throughput, and the amount of information transmittable in the average delay time is also calculated, and then the calculated amount of the information is used as the threshold for frame dropping control. This allows frame dropping control corresponding to the moving video characteristics and the communication condition.
According to a forty-fourth aspect, in the forty-second aspect, when the transmission control portion calculates the average throughput, the time over which average is obtained is variable according to variation of the transmission confirmed number of bits.
As stated above, according to the forty-fourth aspect, decreasing the time for obtaining the average allows frame dropping control quickly responsive to a change in throughput, and in the condition where burst type errors occur, setting the time for obtaining the average longer than the cycle of the occurrence of the burst errors allows frame dropping control to keep stable even with burst errors.
According to a forty-fifth aspect, in the forty-second aspect, the predetermined constant period of time for calculating the average throughput is not less than 300 msec.
As mentioned above, according to the forty-fifth aspect, the time for obtaining the average of 300 msec or longer allows frame dropping control which does not go unstable even with a burst error.
A forty-sixth aspect is directed to a video transmission system in which a video coding device for coding and transmitting a moving picture and a video receiving device for applying certain processing to received video data are connected so that they can communicate,
wherein the video receiving device counts the number of bits of data correctly received in a predetermined constant period of time, calculates an average throughput from the counted bit number and transmits the average throughput to the video coding device, and
the video coding device includes,
a video input portion for forming a video frame from a picture signal and outputting the video frame,
a coding portion for coding the video frame formed in the video input portion,
a transmission buffer for storing the video data of the video frame coded in the coding portion,
a temporary buffer disposed between the coding portion and the transmission buffer for temporarily storing the video data of the video frame coded in the coding portion,
a frame dropping/quantization control portion for determining whether to store into the transmission buffer the video data in the temporary buffer and determining a quantization step size of a video frame coded next, and
a transmission control portion for transmitting the video data in the transmission buffer at a certain transmission rate and outputting the average throughput received from the video receiving device to the frame dropping/quantization control portion,
wherein the frame dropping/quantization control portion,
calculates an amount of the information transmittable in a maximum permissible delay time on the basis of a predetermined maximum permissible delay time and the average throughput, and
performs frame dropping control not to store the video data in the temporary buffer into the transmission buffer if the amount of information of the video frame stored in the temporary buffer is larger than the amount of information transmittable in the maximum permissible delay time and also sets a quantization step size with a larger quantization step size than a quantization step size used to code the video frame stored in the temporary buffer and sends the quantization step size to the coding portion.
As stated above, according to the forty-sixth aspect, the video receiving device calculates the average throughput from the number of bits correctly transmitted in a constant period of time and transmits it to the video coding device. Therefore, even when a plurality of transmission lines having different throughputs exist on the communication line between the two devices, the most suitable frame dropping control can be made considering the throughput between the two devices.
A forty-seventh aspect relates to a video transmission system in which a video coding device coding and transmitting a moving picture and a video receiving device applying certain processing to received video data are communicatably connected, wherein
the video receiving device counts the number of bits of data correctly received in a predetermined constant period of time and transmits the counted bit number to the video coding device, and
the video coding device includes,
a video input portion for forming a video frame from a picture signal and outputting the video frame,
a coding portion for coding the video frame formed in the video input portion,
a transmission buffer for storing the video data of the video frame coded in the coding portion,
a quantization control portion for determining a quantization step size used in coding,
a transmission control portion for transmitting the video data in the transmission buffer at a certain transmission rate and calculating an average throughput from the number of bits transmission confirmed in the predetermined constant period of time and outputting the average throughput to the quantization control portion,
a motion threshold storing portion for storing a motion threshold for determining whether motion of an object is large or small, and
a maximum permissible delay time storing portion for storing a maximum permissible delay time which is a limit of a transmission delay which does not cause visually unnatural impression,
wherein if the amount of generated information per one frame resulting from coding in the coding portion is larger than the motion threshold, the quantization control portion,
obtains a quantization ratio which is a ratio of a quantization step size used to code the video frame exceeding the motion threshold and a quantization step size for which the amount of generated information is to be predicted,
predicts the amount of information generated when the quantization step size is changed from the amount of generated information of the video frame exceeding the motion threshold and the quantization ratio, and
determines a quantization step size transmittable in the maximum permissible delay time from the predicted amount of the generated information, the average throughput, the transmission rate and the maximum permissible delay time and sends the quantization step size to the coding portion, and
the coding portion codes the video frame with the quantization step size sent from the quantization control portion.
As stated above, according to the forty-seventh aspect, the video coding device determines a quantization step size transmittable in the maximum permissible delay time from the predicted amount of the generated information of the video frame with large motion of the object and the average throughput based on the bit number received from the video receiving device. Hence, even if there are a plurality of transmission lines with different throughputs on the communication line between two devices, quantization control most suitable for the throughputs between the two devices can be applied.
According to a forty-eighth aspect, in the forty-seventh aspect, when the transmission control portion calculates the average throughput, the time for averaging is variable according to variation of the number of bits whose transmission are confirmed.
As stated above, according to the forty-eighth aspect, decreasing the time over which the average is obtained allows quantization control quickly responsive to a change in throughput, and in the condition where burst type errors occur, setting the time over which the average is obtained longer than the cycle of the occurrence of the burst errors allows quantization control to keep stable even with burst errors.
According to a forty-ninth aspect, in the forty-seventh aspect, the predetermined constant period of time for calculating the average throughput is not less than 300 msec.
As mentioned above, according to the forty-ninth aspect, the time for obtaining the average of 300 msec or longer permits quantization control which do not go unstable even when a burst error occurs.
A fiftieth aspect is intended for a video transmission system in which a video coding device coding and transmitting a moving picture and a video receiving device applying certain processing to received video data are connected such that they can communicate, wherein
the video receiving device counts the number of bits of data correctly received in a predetermined constant period of time, calculates an average throughput from the counted bit number and transmits the average throughput to the video coding device, and
the video coding device includes,
a video input portion for forming a video frame from a picture signal and outputting the video frame,
a coding portion for coding the video frame formed in the video input portion,
a transmission buffer for storing video data of the video frame coded in the coding portion,
a quantization control portion for determining a quantization step size used in coding,
a transmission control portion for transmitting the video data in the transmission buffer at a certain transmission rate and outputting the average throughput received from the video receiving device to the quantization control portion,
a motion threshold storing portion for storing a motion threshold for determining whether motion of an object is large or small, and
a maximum permissible delay time storing portion for storing a maximum permissible delay time which is a limit of a transmission delay which does not cause visually unnatural impression,
wherein when the amount of generated information per one frame resulting from coding in the coding portion is larger than the motion threshold, the quantization control portion,
obtains a quantization ratio which is a ratio of a quantization step size used to code the video frame exceeding the motion threshold and a quantization step size the amount of generated information for which is to be predicted,
predicts the amount of information generated when the quantization step size is changed from the amount of information of the video frame exceeding the motion threshold and the quantization ratio, and
determines a quantization step size transmittable in the maximum permissible delay time from the predicted amount of the generated information, the average throughput, the transmission rate and the maximum permissible delay time and sends the quantization step size to the coding portion, and
the coding portion codes the video frame with the quantization step size sent from the quantization control portion.
As explained above, according to the fiftieth aspect, the video coding device determines a quantization step size from the predicted amount of the generated information of the video frame with large motion of the object and the average throughput received from the video receiving device. Hence, even if there are a plurality of transmission lines with different throughputs on the communication line between two devices, quantization control most suitable for the throughputs between the two devices can be applied.
A fifty-first aspect relates to a video transmission system in which a video coding device coding and transmitting a moving picture and a video receiving device applying certain processing to received video data are connected so that they can communicate, wherein
the video receiving device counts the number of bits of data correctly received in a predetermined constant period of time and transmits the counted bit number to the video coding device, and
the video coding device includes
a video input portion for forming a video frame from a picture signal and outputting the video frame,
a coding portion for coding the video frame formed in the video input portion,
a transmission buffer for storing video data of the video frame coded in the coding portion,
a quantization control portion for determining a quantization step size used in coding,
a transmission control portion for transmitting the video data in the transmission buffer at a certain transmission rate and calculates an average throughput from the number of bits transmission confirmed in a predetermined constant period of time and outputting the average throughput to the quantization control portion,
a motion threshold storing portion for storing a motion threshold for determining whether motion of an object is large or small, and
an ideal curve storing portion for storing an ideal curve with predetermined operating points most suitable in view of visual properties as a balance between distortion in time due to frame dropping according to a coding rate which is a rate of frames transmittable per unit time and spacial distortion due to a quantization step size,
wherein when the amount of generated information per one frame resulting from coding in the coding portion is larger than the motion threshold, the quantization control portion,
obtains a quantization ratio which is a ratio of a quantization step size used to code the video frame exceeding the motion threshold and a quantization step size for which the amount of generated information is to be predicted,
predicts the amount of information generated when the quantization step size is changed from the amount of generated information of the video frame exceeding the motion threshold and the quantization ratio,
calculates the coding rate from the predicted amount of the generated information, the transmission rate and the average throughput, and
determines a quantization step size most suitable in view of visual properties from the calculated coding rate and the ideal curve and sends the quantization step size to the coding portion, and
the coding portion codes the video frame with the quantization step size sent from the quantization control portion.
As stated above, according to the fifty-first aspect, the video coding device determines the quantization step size most suitable in view of visual properties from the predicted amount of the generated information of the video frame with large motion of the object, the average throughput based on the bit number received from the video receiving device and the ideal curve. Hence, even when there are a plurality of transmission lines with different throughputs on the communication line between two devices, quantization control most suitable for the throughputs between the two devices can be applied.
According to a fifty-second aspect, in the fifty-first aspect, when the transmission control portion calculates the average throughput, the time over which average is obtained is variable according to variation of the number of bits whose transmission are confirmed.
As described above, according to the fifty-second aspect, if the time for obtaining the average is shortened, quantization control can quickly respond to a change in throughput. In the condition where burst type errors occur, if the time for obtaining the average is set longer than the cycle of the occurrence of the burst errors, the quantization control can keep stable even when a burst error occurs.
According to a fifty-third aspect, in the fifty-first aspect, the predetermined constant period of time for calculating the average throughput is not less than 300 msec.
As mentioned above, according to the fifty-third aspect, setting the time for obtaining the average to 300 msec or longer permits quantization control which do not go unstable even when a burst error occurs.
A fifty-fourth aspect relates to a video transmission system in which a video coding device coding and transmitting a moving picture and a video receiving device applying certain processing to received video data are connected so that they can communicate, wherein
the video receiving device counts the number of bits of data correctly received in a predetermined constant period of time and calculates an average throughput from the counted bit number and transmits the average throughput to the video coding device, and
the video coding device includes,
a video input portion for forming a video frame from a picture signal and outputting the video frame,
a coding portion for coding the video frame formed in the video input portion,
a transmission buffer for storing video data of the video frame coded in the coding portion,
a quantization control portion for determining a quantization step size used in coding,
a transmission control portion for transmitting the video data in the transmission buffer at a certain transmission rate and outputting the average throughput received from the video receiving device to the quantization control portion,
a motion threshold storing portion for storing a motion threshold for determining whether motion of an object is large or small, and
an ideal curve storing portion for storing an ideal curve with predetermined operating points most suitable in view of visual properties as a balance between distortion in time due to frame dropping according to a coding rate which is a rate of frames transmittable per unit time and spacial distortion due to the quantization step size,
wherein when the amount of generated information per one frame resulting from coding in the coding portion is larger than the motion threshold, the quantization control portion,
obtains a quantization ratio which is a ratio of a quantization step size used to code the video frame exceeding the motion threshold and a quantization step size the amount of generated information for which is to be predicted,
predicts the amount of information generated when the quantization step size is changed from the amount of generated information of the video frame exceeding the motion threshold and the quantization ratio,
calculates the coding rate from the predicted amount of generated information, the transmission rate and the average throughput, and
determines a quantization step size most suitable in view of the visual properties from the calculated coding rate and the ideal curve and sends the quantization step size to the coding portion, and
the coding portion codes the video frame with the quantization step size sent from the quantization control portion.
As described above, according to the fifty-forth aspect, the video coding device determines a quantization step size from the predicted amount of the generated information of the video frame with large motion of the object and the average throughput received from the video receiving device. Hence, even if there are a plurality of transmission lines with different throughputs on the communication line between the two devices, quantization control most suitable for the throughputs between the two devices can be applied.
A fifty-fifth aspect is directed to a video transmission system in which a video coding device coding and transmitting a moving picture and a video receiving device applying certain processing to received video data are connected through one or more relay stations so that they can communicate,
wherein when detecting an error of the received data, the video receiving device makes automatic retransmission between adjacent one of the relay stations and itself,
the relay station makes automatic retransmission between adjacent one of the relay stations and itself when detecting an error of received data and transmits retransmission occurrence information indicating the occurrence of the retransmission to the video coding device, and
the video coding device includes,
a video input portion for forming a video frame from a picture signal and outputting the video frame,
a coding portion for coding the video frame formed in the video input portion,
a transmission buffer for storing video data coded in the coding portion,
a first quantization control portion for determining a quantization step size most suitable in view of both reproducibility of motion and video quality,
a second quantization control portion for controlling a quantization step size according to communication conditions, and
a transmission control portion for transmitting the video data in the transmission buffer at a certain transmission rate, and if a communication error occurs between adjacent one of the relay stations and the video coding device, making automatic retransmission between the adjacent relay station and the video coding device, and when the transmission control portion itself makes a retransmission, or when it receives the retransmission occurrence information from the video receiving device, sending error information indicating the occurrence of the retransmission to the second quantization control portion,
wherein the second quantization control portion,
when receiving the notice of the error information from the transmission control portion, sets a quantization step size with a larger quantization step size than a quantization step size corresponding to the quantization step size determined in the first quantization control portion and sends the quantization step size to the coding portion, and
when not receiving the notice of the error information from the transmission control portion over a predetermined constant period of time, sends the quantization step size determined in the first quantization control portion unchanged to the coding portion, and
the coding portion codes a video frame coded next with the quantization step size sent from the second quantization control portion.
As explained above, according to the fifty-fifth aspect, the relay station makes automatic retransmission between itself and an adjacent relay station or the video receiving device and transmits the retransmission occurrence information indicative of the occurrence of the retransmission to the video coding device, which permits quantization control taking the communication condition between devices into account.
A fifty-sixth aspect is directed to a video transmission system in which a video coding device coding and transmitting a moving picture and a video receiving device applying certain processing to received video data are connected through one or more relay stations so that they can communicate,
wherein when detecting an error of the received data, the video receiving device makes automatic retransmission between adjacent one of the relay stations and itself,
the relay station makes automatic retransmission between adjacent one of the relay stations and itself when detecting an error of the received data and transmits retransmission occurrence information indicating the occurrence of the retransmission to the video coding device, and
the video coding device includes,
a video input portion for forming a video frame from a picture signal and outputting the video frame,
a coding portion for coding the video frame formed in the video input portion,
a transmission buffer for storing video data of the video frame coded in the coding portion,
a transmission control portion for transmitting the video data in the transmission buffer at a certain transmission rate,
a temporary buffer disposed between the coding portion and the transmission buffer for temporarily storing the video data of the video frame coded in the coding portion, and
a frame dropping/quantization control portion for determining whether to store the video data in the temporary buffer into the transmission buffer and determining a quantization step size for a video frame coded next, wherein
the frame dropping/quantization control portion,
performs threshold control of lowering a frame dropping threshold for determining whether to store the video data in the temporary buffer into the transmission buffer from a predetermined value when receiving notice of the error information from the transmission control portion, and returning the frame dropping threshold to the predetermined value when not receiving the notice of the error information from the transmission control portion in a predetermined constant period of time, and
performs frame dropping control of not storing into the transmission buffer the video data in the temporary buffer when the amount of information of the video frame stored in the temporary buffer is larger than the frame dropping threshold and also sets a quantization step size with a larger quantization step size than a quantization step size used to code the video frame stored in the temporary buffer and sends the quantization step size to the coding portion, and
the coding portion codes the video frame coded next with the quantization step size sent from the frame dropping/quantization control portion.
As stated above, according to the fifty-sixth aspect, the relay station makes automatic retransmission between itself and an adjacent relay station, or the video receiving device and transmits the retransmission occurrence information indicative of the occurrence of the retransmission to the video coding device, which permits coding control taking the communication condition between devices into account.
According to a fifty-seventh aspect, a video coding device coding and transmitting a moving picture includes:
a video coding portion for coding a priority area with a first quantization accuracy and a non-priority area with a second quantization accuracy in a video area to be coded to generate a video coded bit string;
a priority area/non-priority area changing portion for determining and changing the priority area and the non-priority area according to the amount of generated information of the video coded by the video coding portion; and
a transmission control portion for transmitting the video coded bit string.
As mentioned above, according to the fifty-seventh aspect, if the amount of generated information increases or the communication throughput decreases while coding the priority area clearer than the non-priority area, the priority area changing portion reduces the priority area. Hence, the amount of generated information is suppressed and the video quality of the priority area is not largely deteriorated.
A fifty-eighth aspect is directed to a method for controlling quantization operation when coding a video frame in a video coding device in which operating points most suitable in view of visual properties is predetermined as a balance between timewise distortion due to frame dropping according to a coding rate which is a rate of frames transmittable per unit time and spacial distortion due to quantization accuracy when low bit rate compression coding a moving picture and transmitting the coded moving picture through a communication line with a low bit rate, comprising:
a first operation mode in which a quantization accuracy for next coding is determined according to the relation in magnitude between a coding rate resulting from coding an entire video frame with a certain quantization accuracy and a coding rate at a point most suitable in view of the visual properties corresponding to that quantization accuracy; and
a second operation mode in which a video area to be coded is divided into a priority area and a non-priority area, the priority area being coded with a relatively high first quantization accuracy and the non-priority area being coded with a relatively low second quantization accuracy, and the quantization accuracy and sizes of the priority area and the non-priority area for the next coding are determined according to the relation in magnitude between coding rate resulting from coding with the first and second quantization accuracies and a coding rate at a point most suitable in view of the visual properties corresponding to the quantization accuracies,
wherein when the quantization accuracy determined in the first operation mode falls below a predetermined permissible lower limit value of the quantization accuracy, the operation mode moves from the first operation mode to the second operation mode, and
when the size of the priority area determined in the second operation mode reaches its maximum, the operation mode moves from the second operation mode to the first operation mode.
As stated above, according to the fifty-eighth aspect, when the spacial video quality absolutely due to an error in quantization accuracy and corresponding frame jump are in the permissible range, the conventional quantization control considering the visual properties is made. When the spacial video quality and the frame dropping exceed the permissible limit, the video quality of the spacial area which is not very important in the video is sacrificed so that quantization control considering the visual properties can be applied to the important spacial area. Then, coding parameters for the changing input video can be kept at values at which the balance between the reproducibility of motion and the spacial resolution is the most suitable in view of the visual properties.
According to a fifty-ninth aspect, in the fifty-eighth aspect, the coding rates most suitable in view of the visual properties corresponding to each of the quantization accuracies are each determined as a set of points in a region surrounded by an upper limit value and a lower limit value,
in the first and second operation modes,
the quantization accuracy for the next coding is increased when the coding rate resulting from coding with a certain quantization accuracy exceeds the upper limit value,
the quantization accuracy for the next coding is decreased when the coding rate resulting from coding with a certain quantization accuracy falls below the lower limit value, and
when the coding rate resulting from coding with a certain quantization accuracy is within the region between the upper limit value and the lower limit value, the quantization accuracy for the next coding is not changed.
As stated above, according to the fifty-ninth aspect, most stable coding control can be realized even when the input video quickly changes by allowing some latitude to the visually most suitable coding rate for each quantization accuracy.
According to a sixtieth aspect, in the fifty-ninth aspect, a minimum size of the quantization accuracy when changed in the first and second operation modes is chosen to a size not less than a value at which a difference of spacial distortion is visually recognizable.
As stated above, according to the sixtieth aspect, the change of the quantization accuracy given per unit time is increased so that the change of the visually most suitable operating point due to a change of input video can be quickly followed.
According to a sixty-first aspect, in the third or fourth aspect, when the transmission control portion calculates the average throughput, the time over which average is obtained is variable according to variation of the number of bits whose transmission are confirmed.
As mentioned above, according to the sixty-first aspect, decreasing the time for obtaining the average allows quantization control and frame dropping control quickly responsive to a change in throughput, and in the condition where burst type errors occur, setting the time for obtaining the average longer than the cycle of the occurrence of the burst errors allows quantization control and frame dropping control to keep stable even with burst errors.
According to a sixty-second aspect, in the third or fourth aspect, the predetermined constant period of time for calculating the average throughput is not less than 300 msec.
As mentioned above, according to the sixty-second aspect, the time for obtaining the average of 300 msec or longer permits quantization control and frame dropping control which do not go unstable even when a burst error occurs.
According to a sixty-third aspect, in the seventh or eighth aspect, when the transmission control portion calculates the average throughput, the time for obtaining the average is variable according to variation of the number of bits whose transmission are confirmed.
As mentioned above, according to the sixty-third aspect, decreasing the time over which the average is obtained allows quantization control and frame dropping control quickly responsive to a change in throughput, and in the condition where burst type errors occur, setting the time over which the average is obtained longer than the cycle of the occurrence of the burst errors allows quantization control and frame dropping control to keep stable even with burst errors.
According to a sixty-fourth aspect, in the seventh or eighth aspect, the predetermined constant period of time for calculating the average throughput is not less than 300 msec.
As mentioned above, according to the sixty-fourth aspect, the time for obtaining the average of 300 msec or longer permits quantization control and frame dropping control which do not go unstable even with occurrence of a burst error.
According to a sixty-fifth aspect, in the forty-sixth aspect, the frame dropping/quantization control portion, further,
calculates an average delay time on the basis of the average amount of the generated information of the video frame transmitted in a constant period of time and the average throughput, and
uses the calculated average delay time as the maximum permissible delay time.
As mentioned above, according to the sixty-fifth aspect, the average delay time is calculated from the average amount of generated information of video frame transmitted in a constant period of time and the average throughput and the amount of information transmittable in the average delay time is also calculated. Then the calculated amount of information is used as the threshold for the frame dropping control. This allows frame dropping control corresponding to the moving video characteristics and communication condition.
According to a sixty-sixth aspect, in the fourth-sixth aspect, when the video receiving device calculates the average throughput, the time over which average is obtained is variable according to variation of the number of bits of the data correctly received.
As mentioned above, according to the sixty-sixth aspect, decreasing the time over which the average is obtained allows frame dropping control quickly responsive to a change in throughput, and in the condition where burst type errors occur, setting the time over which the average is obtained longer than the cycle of the occurrence of the burst errors allows frame dropping control to keep stable even with burst errors.
According to a sixty-seventh aspect, in the forty-sixth aspect, the predetermined constant period of time for calculating the average throughput is not less than 300 msec.
As mentioned above, according to the sixty-seventh aspect, the time for obtaining the average of 300 msec or longer permits frame dropping control which do not go unstable even with occurrence of a burst error.
According to a sixty-eighth aspect, in the fiftieth aspect, when the video receiving device calculates the average throughput, the time over which the average is obtained is variable according to variation of the number of bits of data correctly received.
As mentioned above, according to the sixty-eighth aspect, decreasing the time over which the average is obtained allows quantization control quickly responsive to a change in throughput, and in the condition where burst type errors occur, setting the time over which the average is obtained longer than the cycle of the occurrence of the burst errors allows quantization control to keep stable even with burst errors.
According to the sixty-ninth aspect, in the fiftieth aspect, the predetermined constant period of time for calculating the average throughput is not less than 300 msec.
As mentioned above, according to the sixty-ninth aspect, the time for obtaining the average of 300 msec or longer permits quantization control which do not go unstable even with occurrence of a burst error.
According to a seventieth aspect, in the fifty-fourth aspect, when the video receiving device calculates the average throughput, the time over which average is obtained is variable according to variation of the number of bits of correctly received data.
As mentioned above, according to the seventieth aspect, decreasing the time over which the average is obtained allows quantization control quickly responsive to a change in throughput, and in the condition where burst type errors occur, setting the time over which the average is obtained longer than the cycle of the occurrence of the burst error allows quantization control to keep stable even with burst errors.
According to a seventy-first aspect, in the fifty-fourth aspect, the predetermined constant period of time for calculating the average throughput is not less than 300 msec.
As mentioned above, according to the seventy-first aspect, since the time for obtaining the average is set to 300 msec or longer, determination of the quantization step size which does not become unstable even when a burst error occurs is realized when the movement of the object is large.
According to a seventy-second aspect, in the tenth aspect, when the quantization control portion calculates the average throughput, a time for calculating the average is variable according to whether burst errors occur or not.
As stated above, according to the seventy-second aspect, decreasing the time over which the average is obtained permits the quantization control and the frame dropping control quickly responsive to a change in throughput, and in the condition where burst type errors occur, setting the time over which the average is obtained larger than the cycle of the occurrence of the burst errors allows quantization control and frame dropping control to keep stable even with burst errors.
According to a seventy-third aspect, in the eleventh aspect, when the quantization control portion calculates the average throughput, the time for calculating the average is variable according to whether burst errors occur or not.
As stated above, according to the seventy-third aspect, decreasing the time for calculating the average allows the quantization control and the frame dropping control quickly responsive to a change in throughput, and in the condition where burst type errors occur, setting the time over which the average is obtained larger than the cycle of the occurrence of the burst errors allows the quantization control and the frame dropping control to keep stable even with burst errors.
According to a seventy-fourth aspect, in the tenth aspect, the predetermined certain time for calculating the average throughput is not less than 300 msec.
As stated above, according to the seventy-fourth aspect, since the time for obtaining the average is set to 300 msec or longer, the quantization control and the frame dropping control can keep stable even with burst errors.
According to a seventy-fifth aspect, in the eleventh aspect, the predetermined certain time for calculating the average throughput is not less than 300 msec.
As stated above, according to the seventy-fifth aspect, since the time for obtaining the average is set to 300 msec or longer, the quantization control and the frame dropping control can keep stable even with burst errors.
According to a seventy-sixth aspect, in the fourth aspect, when the transmission control portion calculates the average throughput, the time for calculating the average is variable according to whether burst errors occur or not.
As stated above, according to the seventy-sixth aspect, decreasing the time over which the average is obtained permits the quantization control and the frame dropping control quickly responsive to a change in throughput, and in the condition where burst type errors occur, setting the time over which the average is obtained larger than the cycle of the occurrence of the burst errors allows the quantization control and the frame dropping control to keep stable even with burst errors.
According to a seventy-seventh aspect, in the fourth aspect, the predetermined certain time for calculating the average throughput is not less than 300 msec.
As stated above, according to the seventy-seventh aspect, since the time for obtaining the average is set to 300 msec or longer, quantization control and frame dropping control can keep stable even with burst errors.
According to a seventy-eighth aspect, in the eighth aspect, when the transmission control portion calculates the average throughput, the time for calculating the average is variable according to whether burst errors occur or not.
As stated above, according to the seventy-eighth aspect, decreasing the time over which the average is obtained allows quantization control and frame dropping control quickly responsive to a change in throughput, and in the condition where burst type errors occur, setting the time over which the average is obtained larger than the cycle of the occurrence of the burst errors allows the quantization control and the frame dropping control to keep stable even with burst errors.
According to a seventy-ninth aspect, in the eighth aspect, the predetermined certain time for calculating the average throughput is not less than 300 msec.
As stated above, according to the seventy-ninth aspect, since the time for obtaining the average is set to 300 msec. or longer, quantization control and frame dropping control can keep stable even with burst errors.
According to an eightieth aspect, in any of the third to fourth, seventh to thirteenth, fifteenth to seventeenth, nineteenth to twenty-first, sixty-first to sixty-fourth, or seventy-second to seventy-ninth aspects, the average throughput for the certain time is calculated on the basis of the amount of data outputted in a unit time t from the transmission buffer which temporarily stores the coded data.
As stated above, according to the eightieth aspect, since the average throughput is calculated based on the number of bits read from the transmission buffer, Go-back-N or Selective Reject can be adopted as a retransmission method.
According to an eighty-first aspect, in any of the third to fourth, seventh to thirteenth, fifteenth to seventeenth, nineteenth to twenty-first, sixty-first to sixty-fourth, or seventy-second to seventy-ninth aspects, the average throughput for the certain time is calculated as each of the certain time passes, by dividing by the certain time the amount of data outputted for the certain time, which is obtained by accumulating the amount of data outputted in the unit time t for the certain time, as each of the certain time passes.
As stated above, in the eighty-first aspect, a mean value is calculated by dividing the total number of bits for the certain time by the certain time for each of the certain time and outputted for each of the certain time, which permits calculation of the average throughput by a simple process and furthermore, adoption of Go-back-N or Selective Reject as a retransmission method.
According to an eighty-second aspect, in any of the third to fourth, seventh to thirteenth, fifteenth to seventeenth, nineteenth to twenty-first, sixty-first to sixty-fourth, or seventy-second to seventy-ninth aspects, the average throughput for the certain time is calculated as each of a unit time t passes, by dividing by the certain time the amount of data outputted in the certain time, which is the result of accumulating the amount of data outputted in the unit time t between a present time and the certain time prior to the present time, as each of the unit time passes.
As stated above, according to the eighty-second aspect, the mean value is calculated by dividing the total number of bits for the certain time by the certain time for each of the unit time t and outputted for each of the unit time t. This allows the average throughput to be up dated for each of the unit time t and permits careful control. Furthermore, Go-back-N or Selective Reject can be adopted as the retransmission method.
According to an eighty-third aspect, in any of the third to fourth, seventh to thirteenth, fifteenth to seventeenth, nineteenth to twenty-first, sixty-first to sixty-fourth, or seventy-second to seventy-ninth aspects, the average throughput for the certain time is calculated on the basis of the amount of data outputted in a unit time t which is equal to a video frame period of the coded video from the transmission buffer temporarily storing the coded data.
As stated above, according to the eighty-third aspect, equating the unit time t with the video frame period of the coded video permits synchronization with a coding timing.
According to an eighty-fourth aspect, in any of the third to fourth, seventh to thirteenth, fifteenth to seventeenth, nineteenth to twenty-first, sixty-first to sixty-fourth, or seventy-second to seventy-ninth aspects, the average throughput for the certain time is calculated as each of the certain time passes, by dividing by the certain time the amount of data outputted for the certain time, which is obtained by accumulating the amount of data outputted in the unit time t equal to a video frame period of coded video for the certain time for the certain time, as each of the certain time passes.
As mentioned above, according to the eighty-fourth aspect, equating the unit time t with the video frame period of the coded video permits synchronization with the coding timing. Furthermore, the mean value is calculated by dividing the total number of bits for the certain time by the certain time for each of the certain time and outputted for each of the certain time, resulting in calculating the average throughput by the simple process.
According to an eighty-fifth aspect, in any of the third to fourth, seventh to thirteenth, fifteenth to seventeenth, nineteenth to twenty-first, sixty-first to sixty-fourth, or seventy-second to seventy-ninth aspects, as each of the unit time t equal to the video frame period of the coded video passes, by dividing by the certain time the amount of data outputted for the certain time, which is the result of accumulating the amount of data outputted in the unit time t between the certain time prior to the present time to the present time, the average throughput for the certain time is calculated as each of the unit time t equal to the video frame period of the coded video passes.
As mentioned above, according to the eighty-fifth aspect, equating the unit time t with the video frame period of the coded video permits synchronization with the coding timing. Furthermore, the mean value is calculated by dividing the total number of bits for the certain time by the certain time for each of the unit time t and outputted for each of the unit time t, which makes it possible to update the average throughput for each of the unit time t and make careful control.
According to an eighty-sixth aspect, in any of the third to fourth, seventh to thirteenth, fifteenth to seventeenth, nineteenth to twenty-first, sixty-first to sixty-fourth, or seventy-second to seventy-ninth aspects, the average throughput for the certain time is calculated on the basis of the amount of data outputted in a unit time t equal to a one frame time of the transmitted frames from the transmission buffer temporarily storing the coded data.
As mentioned above, according to the eighty-sixth aspect, equating the unit time t with the one frame time of the transmission frames makes it possible to readily synchronize with a transmission timing of the transmitted frames and furthermore, adopt Go-back-N or Selective Reject as the retransmission method.
According to an eighty-seventh aspect, in any of the third to fourth, seventh to thirteenth, fifteenth to seventeenth, nineteenth to twenty-first, sixty-first to sixty-fourth, or seventy-second to seventy-ninth aspects, the average throughput for the certain time is calculated as each of the certain time passes, by dividing by the certain time the amount of data outputted for the certain time, which is the result of accumulating for the certain time the amount of data outputted in the unit time t equal to the one frame time of the transmitted frames, as each of the certain time passes.
As mentioned above, according to the eighty-seventh aspect, equating the unit time t with the one frame time of the transmission frames easily permits synchronization with the transmission timing of the transmitted frames. Furthermore, the mean value is calculated by dividing the total number of bits for the certain time by the certain time for each of the certain time and outputted for each of the certain time, resulting in calculating the average throughput by the simple process.
According to an eighty-eighth aspect, in any of the third to fourth, seventh to thirteenth, fifteenth to seventeenth, nineteenth to twenty-first, sixty-first to sixty-fourth, or seventy-second to seventy-ninth aspects, as each of the unit time t equal to the one frame time of the transmitted frames passes, by dividing by the certain time the amount of data outputted for the certain time, which is the result of accumulating the amount of data outputted in the unit time t between the certain time prior to the present time to the present time, the average throughput for the certain time is calculated as each of the unit time t equal to the one frame time of the transmitted frames passes.
As mentioned above, according to the eighty-eighth aspect, equating the unit time t with the one frame time of the transmitted frames easily permits synchronization with the transmission timing of the transmission frames. Furthermore, the mean value is calculated by dividing the total number of bits for the certain time by the certain time for each of the unit time t and outputted for each of the unit time t, which makes it possible to update the average throughput for each of the unit time t and achieve careful control.
According to an eighty-ninth aspect, in any of the third to fourth, seventh to thirteenth, fifteenth to seventeenth, nineteenth to twenty-first, sixty-first to sixty-fourth, or seventy-second to seventy-ninth aspects, the average throughput for the certain time is calculated on the basis of the amount of data outputted for a unit time t which is equal to a least common multiple of the video frame period of the coded frames and the one frame time of the transmission frames from the transmission buffer temporarily storing the coded data.
As mentioned above, according to the eighty-ninth aspect, setting the unit time t to the least common multiple of the transmission frame time and the period of video makes it possible to readily synchronize with both of them and furthermore, adopt Go-back-N or Selective Reject as the retransmission method.
According to a ninetieth aspect, in any of the third to fourth, seventh to thirteenth, fifteenth to seventeenth, nineteenth to twenty-first, sixty-first to sixty-fourth, or seventy-second to seventy-ninth aspects, the average throughput for the certain time is calculated as each of the certain time passes, by dividing by the certain time the amount of data outputted for the certain time which is obtained by accumulating for the certain time the amount of data outputted in the unit time t equal to the least common multiple of the video frame period of the coded video and the one frame time of the transmission frames.
As stated above, according to the ninetieth aspect, setting the unit time t to the least common multiple of the transmission frame time and the period of the video easily permits synchronization with both of them. Moreover, the mean value is calculated by dividing the total number of bits for the certain time by the certain time for each of the certain time and outputted for each of the certain time, resulting in calculation of the average throughput by the simple process.
According to a ninety-first aspect, in any of the third to fourth, seventh to thirteenth, fifteenth to seventeenth, nineteenth to twenty-first, sixty-first to sixty-fourth, or seventy-second to seventy-ninth aspects, as each of the unit time t equal to the least common multiple of the video frame period of the coded video and the one frame time of the transmitted frames passes, by dividing by the certain time the amount of data outputted for the certain time, which is the result of accumulating the amount of data outputted in the unit time t between the certain time before the present time and the present time, the average throughput for the certain time is calculated as each of the unit time t passes.
As described above, in the ninety-first aspect, setting the unit time t to the least common multiple of the transmission frame time and the period of the video easily permits synchronization with both of them. Moreover, the mean value is calculated by dividing the total number of bits for the certain time by the certain time for each of the unit time t and outputted for each of the unit time t, which makes it possible to update the average throughput for each of the unit time t and make careful control.
According to a ninety-second aspect, in the thirty-ninth aspect, when the video receiving device calculates the average throughput, the time for calculating the average is variable according to variation of the number of bits of correctly received data.
As described above, according to the ninety-second aspect, decreasing the time over which the average is obtained allows quantization control quickly responsive to a change in throughput, and in the condition where burst type errors occur, setting the time over which the average is obtained larger than the cycle of the occurrence of the burst errors allows quantization control to keep stable even with burst errors.
According to a ninety-third aspect, in the thirty-ninth aspect, the predetermined certain time for calculating the average throughput is not less than 300 msec.
As stated above, according to the ninety-third aspect, setting the time for obtaining the average to 300 msec. or longer enables to keep quantization control stable even with burst errors.
According to a ninety-fourth aspect, an average throughput for a certain time T is calculated on the basis of an amount of data outputted in an unit time t from a transmission buffer temporarily storing coded data.
As described above, according to the ninety-fourth aspect, since the average throughput is calculated on the basis of the number of bits read from the transmission buffer, Go-back-N or Selective Reject can be adopted as the retransmission method.
According to a ninety-fifth aspect, in the ninety-fourth aspect, a period in which the amount of data outputted in the unit time t is shorter than a predetermined threshold is detected from the transmission buffer and taken as a certain time T.
As stated above, according to the ninety-fifth aspect, taking the period of burst errors as the certain time T enables to calculate the average throughput keeping stable with the occurrence of burst errors.
According to a ninety-sixth aspect, in the ninety-fourth aspect, the average throughput for the certain time T is calculated as each of the certain time T passes, by dividing by the certain time T the amount of data outputted for the certain time T which is obtained by accumulating the amount of data outputted in the unit time t for the certain time T, as each of the certain time T passes.
As stated above, in the ninety-sixth aspect, a mean value is calculated by dividing the total number of bits for the certain time T by the certain time T for each of the certain time T and outputted for each of the certain time T, which enables to calculate the average throughput by the simple process.
According to a ninety-seventh aspect, in the ninety-sixth aspect, the period in which the amount of data outputted in the unit time t is smaller than the predetermined threshold is detected from the transmission buffer and taken as the certain time T.
As stated above, according to the ninety-seventh aspect, taking the period of burst errors as the certain time T makes it possible to calculate the average throughput keeping stable with the occurrence of burst errors.
According to a ninety-eighth aspect, in the ninety-fourth aspect, the average throughput for the certain time T is calculated as each of the unit time t passes, by dividing by the certain time T the amount of data outputted for the certain time T, which is obtained by accumulating the amount of data outputted in the unit time t between the present time and the certain time T prior to the present time, as each of the unit time passes.
As stated above, according to the ninety-eighth aspect, the mean value is calculated by dividing the total number of bits for the certain time T by the certain time T for each of the unit time t and outputted for each of the unit time t, which allows the average throughput to be updated for each of the unit time t and careful control to be performed.
According to a ninety-ninth aspect, in the ninety-eighth aspect, the period in which the amount of data outputted in the unit time t is smaller than the predetermined threshold is detected from the transmission buffer and taken as the certain time T.
As stated above, according to the ninety-ninth aspect, taking the period of burst errors as the certain time T makes it possible to calculate the average throughput keeping stable with the occurrence of burst errors.
According to a one-hundred-th aspect, in the ninety-fourth to ninety-ninth aspects, the unit time t is taken as the video frame period of the coded video.
As stated above, according to the one-hundred-th aspect, equating the unit time t with the video frame period of the coded video permits synchronization with the coding timing.
According to a one-hundred-first aspect, in the ninety-fourth to ninety-ninth aspect, the unit time t is taken as the one frame time of the transmission frames.
As stated above, according to the one-hundred-first aspect, taking the unit time t as the one frame time of the transmission frames easily permits synchronization with the transmission timing of the transmission frames.
According to a one-hundred-second aspect, in the ninety-fourth to ninety-ninth aspects, the unit time t is taken as a least common multiple of the video frame period of the coded video and the one frame time of the transmission frames.
As mentioned above, according to the one-hundred-second aspect, taking the unit time t as the least common multiple of the transmission frame time and the period of video readily permits synchronization with both of them.
According to a one-hundred-third aspect, in any of the twelfth, sixteenth, twentieth, sixty-first, sixty-third, seventy-second to seventy-third, seventy-fifth, or seventy-eighth aspects, the period in which the amount of data outputted in the unit time t is smaller than the predetermined threshold is detected from the transmission buffer temporarily storing the coded data and taken as an average time used in calculating the average throughput.
As mentioned above, according the one-hundred-third aspect, taking the period of burst errors as the averaging time used in obtaining the average throughput allows the quantization control and the frame dropping control to keep stable even with the occurrence of burst errors.
According to a one-hundred-fourth aspect, in any of the twelfth, sixteenth, twentieth, sixty-first, sixty-third, seventy-second to seventy-third, seventy-sixth or seventy-eighth aspects, the period in which the amount of data outputted in the unit time t equal to the one frame time of the transmission frames is smaller than the predetermined threshold is detected from the transmission buffer temporarily storing the coded data and taken as the averaging time used in calculating the average throughput.
As mentioned above, according to the one-hundred-fourth aspect, taking the period of burst errors as the averaging time used in obtaining the average throughput allows the quantization control and the frame dropping control to keep stable even with the occurrence of burst errors.
According to a one-hundred-fifth aspect, in any of the twelfth, sixteenth, twentieth, sixty-first, sixty-third, seventy-second to seventy-third, seventy-sixth or seventy-eighth aspects, the period in which the amount of data outputted in the unit time t equal to the video frame period of the coed video is smaller than the predetermined threshold is detected from the transmission buffer temporarily storing the coded data and taken as the averaging time used in calculating the average throughput.
As mentioned above, according to the one-hundred-fifth aspect, taking the period of burst errors as the averaging time used in obtaining the average throughput allows the quantization control and the frame dropping control to keep stable even with burst errors.
According to a one-hundred-sixth aspect, in any of the twelfth, sixteenth, twentieth, sixty-first, sixty-third, seventy-second to seventy-third, seventy-sixth or seventy-eighth aspects, the period in which the amount of data outputted in the unit time t equal to the least common multiple of the video frame period of the coded video and the one frame time of the transmission frames is smaller than the predetermined threshold is detected from the transmission buffer temporarily storing the coded data and taken as the averaging time used in calculating the average throughput.
As mentioned above, according to the one-hundred-sixth aspect, taking the period of burst errors as the averaging time used in obtaining the average throughput allows the quantization control and the frame dropping control to keep stable even with burst errors.
According to a one-hundred-seventh aspect, in the thirty-nine aspect, when the video receiving device calculates the average throughput, the time for calculating the average is variable according to variation of the number of bits correctly received data.
As mentioned above, according to the one-hundred-seventh aspect, decreasing the time over which the average is obtained permits the quantization control quickly responsive to a change in throughput, and in the condition where burst type errors occur, setting the time over which the average is obtained larger than the cycle of the occurrence of the burst errors allows the quantization control to keep stable even with burst errors.
According to a one-hundred-eighth aspect, in the thirty-nine aspect, the predetermined certain time for calculating the average throughput is not less than 300 msec.
As mentioned above, according to the one-hundred-eighth aspect, setting the time for calculating the average throughput not less than 300 msec allows the quantization control to keep stable even with burst errors.
According to a one-hundred-ninth aspect, in any of the thirty-sixth to thirty-eighth, forty-second to forty-fifth, forty-seventh to forty-ninth, or fifty-first to fifty-third aspects, the number of bits received without errors for the certain time, which is obtained by accumulating for the certain time the number of bits received without errors for a predetermined unit time t as each of the certain time passes, is sent to the video coding device as each of the certain time passes.
As stated above, according to the one-hundred-ninth aspect, the number of bits received without errors for the certain time can be calculated by a simple process and in the video coding device the average throughput for each of the certain time can be obtained.
According to a one-hundred-tenth aspect, in any of the thirty-ninth, forty-sixth, fiftieth, fifty-fourth, sixty-fifth to seventy-first, or one-hundred-seventh to one-hundred-eighth aspects, as each of the certain time passes, by dividing by the certain time the number of bits received without errors for the certain time, which is obtained by accumulating for the certain time the number of bits received without errors for a predetermined unit time t, the average throughput for the certain time is sent to the video coding device as each of the certain time passes.
As stated above, in the one-hundred-tenth aspect, the average throughput for the certain time can be calculated by the simple process and in the video coding device the average throughput for each of the certain time can be obtained.
According to a one-hundred-eleventh aspect, in any of the thirty-sixth to thirty-eighth, forty-second to forty-fifth, forty-seventh to forty-ninth, or fifty-first to fifty-third aspects, the number of bits received without errors for the certain time, which is obtained by,as each of a predetermined unit time t passes, accumulating the number of bits received without errors for the unit time t between the certain time T before a present time and the present time, is sent to the video coding device as each of the unit time t passes.
As stated above, according to the one-hundred-eleventh aspect, sending for each of a unit time t the number of bits received without errors for the certain time to the video coding device makes it possible to obtain the average throughput for each of the unit time t and perform careful control in the video coding device.
According to an one-hundred-twelfth aspect, in any of the thirty-ninth, forty-sixth, fiftieth, fifty-fourth, sixty-fifth to seventy-first, or one-hundred-seventh to one-hundred-eighth aspects, as each of a predetermined unit time passes, by dividing by the certain time the bit number received without errors for the certain time, which is obtained by accumulating the bit number received without errors for the unit time t between the certain time before a present time and the present time, the average throughput for the certain time is sent to the video coding device as each of the unit time t passes.
As stated above, in the one-hundred-twelfth aspect, transmitting the average throughput for the certain time to the video coding device for each of the unit time t permits careful control in the video coding device.
According to an one-hundred-thirteenth aspect, in any of the thirty-sixth to thirty-eighth, forty-second to forty-fifth, forty-seventh to forty-ninth, or fifty-first to fifty-third aspects, the video receiving device has the same function of coding to transmit videos as the video coding device does, and the bit number received without errors for the certain time, which is obtained by,as each of the certain time passes, accumulating for the certain time the bit number received without errors for a unit time t equal to a video frame period of the coded video, is transmitted to the video coding device as each of the certain time passes.
As stated above, in the one-hundred-thirteenth aspect, equating the unit time t with the video frame period of the coded videos readily permits synchronization with a coding timing. Furthermore, the bit number received without errors for the certain time can be calculated by a simple process and the average throughput for the certain time can be obtained in the video coding device.
According to a one-hundred-fourteenth aspect, in any of the thirty-ninth, the forty-sixth, the fiftieth, the fifty-fourth, the sixty-fifth to the seventy-first, or the one-hundred-seventh to the one-hundred-eighth aspects, the video receiving device has the same function of coding to transmit videos as the video coding device does, and, as each of the constant period of time passes, by dividing by the constant period of time the bit number received without errors for the constant period of time, which is the result of accumulating the bit number received without errors for the unit time t equal to the video frame period of the coded videos for the constant period of time, an average throughput for the constant period of time is sent to the video coding device as each of the constant period of time passes.
As stated above, according to the one-hundred-fourteenth aspect, equating the unit time t with the video frame period of the coded videos easily enables synchronization with the coding timing. Furthermore, the average throughput for the constant period of time can be calculated with simple processing, and the average throughput for each of the constant period of time can be obtained in the video coding device.
According to a one-hundred-fifteenth aspect, in any of the thirty-sixth to the thirty-eighth, the forty-second to the forty-fifth, the forty-seventh to the forty-ninth, or the fifty-first to the fifty-third aspects, the video receiving device has the same function of coding to transmit videos as the video coding device does, and the bit number received without errors for the constant period of time, which is the result of, as each of the unit time t equal to the video frame period of the coded videos passes, accumulating the bit number received without errors for the unit time t which is between the constant period of time before the present time and the present time, is sent to the video coding device as each of the unit time t passes.
As stated above, according to the one-hundred-fifteenth aspect, equating the unit time t with the video frame period of the coded videos easily permits synchronization with the code timing. Furthermore, sending the bit number received without errors for the constant period of time to the video coding device for each of the unit time t allows the video coding device to obtain the average throughput for each of the unit time t and careful control can be realized.
According to a one-hundred-sixteenth aspect, in the thirty-ninth, the forty-sixth, the fiftieth, the fifty-fourth, the sixty-fifth to the seventy-first, or the one-hundred-seventh to the one-hundred-eighth aspects, the video receiving device has the same function of coding to transmit videos as the video coding device does, and, as each of the unit time t equal to the video frame period of the coded videos passes, by dividing by the constant period of time the bit number received without errors for the constant period of time, which is the result of accumulating the bit number received without errors for the unit time t which is between the constant period of time prior to the present time and the present time, the average throughput for the constant period of time is sent to the video coding device as each of the unit time t passes.
As stated above, according to the one-hundred-sixteenth aspect, equating the unit time t with the video frame period of the coded videos easily enables synchronization with the code timing. Furthermore, sending the average throughput for the constant period of time to the video coding device for each of the unit time t enables careful control in the video coding device.
According to a one-hundred-seventeenth aspect, in any of the thirty-sixth to the thirty-eighth, the forty-second to the forty-fifth, the forty-seventh to the forty-ninth, or the fifty-first to the fifty-third aspects, the bit number received without errors for the constant period of time, which is the result of, as each of the constant period of time passes, accumulating the bit number received without errors for the unit time t equal to the one frame time of the transmitted frames for the constant period of time is sent to the video coding device as each of the constant period of time passes.
As stated above, according to the one-hundred-seventeenth aspect, equating the unit time t with the one frame time of the transmitted frames easily enables synchronization with the transmission timing of the transmitted frames. Furthermore, the bit number received without errors for the constant period of time can be calculated with simple processing and the average throughput for each of the constant period of time can be obtained in the video coding device.
According to a one-hundred-eighteenth aspect, in any of the thirty-ninth, the forty-sixth, the fiftieth, the fifty-fourth, the sixty-fifth to the seventy-first, or the one-hundred-seventh to the one-hundred-eighth aspects, as each of the constant period of time passes, by dividing by the constant period of time the bit number received without errors for the constant period of time, which is the result of accumulating the bit number received without errors for the unit time t equal to the one frame time of the transmitted frames for the constant period of time, the average throughput for the constant period of time is sent to the video coding device as each of the constant period of time passes.
As stated above, according to the one-hundred-eighteenth aspect, equating the unit time t with the one frame time of the transmitted frames easily permits synchronization with the transmission timing of the transmitted frames. Furthermore, the average throughput for the constant period of time can be calculated with simple processing, and the average throughput for each of the constant period of time can be obtained in the video coding device.
According to a one-hundred-nineteenth aspect, in any of the thirty-sixth to the thirty-eighth, the forty-second to the forty-fifth, the forty-seventh to the forty-ninth, or the fifty-first to the fifty-third aspects, the bit number received without errors for the constant period of time, which is the result of, as each of the unit time t equal to the one frame time of the transmitted frames passes, accumulating the bit number received without errors for the unit time t between the constant period of time before the present time and the present time, is sent to the video coding device as each of the unit time t passes.
As stated above, according to the one-hundred-nineteenth aspect, equating the unit time t with the one frame time of the transmitted frames easily permits synchronization with the transmission timing of the transmitted frames. Furthermore, by sending the bit number received without errors for the constant period of time for each of the unit time t, the average throughput can be obtained for each of the unit time t and careful control can be realized in the video coding device.
According to a one-hundred-twentieth aspect, in any of the thirty-ninth, the forty-sixth, the fiftieth, the fifty-fourth, the sixty-fifth to the seventy-first, or the one-hundred-seventh to the one-hundred-eighth aspects, as each of the unit time t equal to the one frame period of the transmitted frames passes, by dividing by the constant period of time the bit number received without errors for the constant period of time, which is obtained by accumulating the bit number received without errors for the unit time t between the constant period of time before the present time and the present time, the average throughput for the constant period of time is sent to the video coding device as each of the unit time t passes.
As stated above, according to the one-hundred-twentieth aspect, equating the unit time t with the one frame time of the transmitted frames easily enables synchronization with the transmission timing of the transmitted frames. Furthermore, sending the average throughput for the constant period of time for each of the unit time t to the video coding device permits careful control in the video coding device.
According to a one-hundred-twenty-first aspect, in any of the thirty-sixth to the thirty-eighth, the forty-second to the forty-fifth, the forty-seventh to the forty-ninth, or the fifty-first to the fifty-third aspects, the video receiving device has the same function of coding to transmit videos as the video coding device does, and the bit number received without errors for the constant period of time, which is obtained, as each of the constant period of time passes, by accumulating the bit number received without errors for the unit time t equal to the least common multiple of the video frame period of the coded videos and the one frame time of the transmitted frames for the constant period of time, is sent to the video coding device as each of the constant period of time passes.
As stated above, according to the one-hundred-twenty-first aspect, taking the unit time t as the least common multiple of the transmission frame time and the period of videos easily enables synchronization with both of them. Furthermore, the bit number received without errors for the constant period of time can be calculated with simple processing, and the average throughput for each of the constant period of time can be obtained in the video coding device.
According to a one-hundred-twenty-second aspect, in any of the thirty-ninth, the forty-sixth, the fiftieth, the fifty-fourth, the sixty-fifth to the seventy-first, or the one-hundred-seventh to the one-hundred-eighth aspects, the video receiving device has the same function of coding to transmit videos as the video coding device does, and, as each of the constant period of time passes, by dividing by the constant period of time the bit number received without errors for the constant period of time, which is the result of accumulating the bit number received without errors for the unit time t equal to the least common multiple of the video frame period of the coded videos and the one frame time of the transmitted frames for the constant period of time, the average throughput for the constant period of time is transmitted to the video coding device as each of the constant period of time passes.
As stated above, according to the one-hundred-twenty-second aspect, taking the unit time t as the least common multiple of the transmission frame period and the period of videos easily enables synchronization with the both. Furthermore, the average throughput for the constant period of time can be calculated with simple processing, and the average throughput for each of the constant period of time can be obtained in the video coding device.
According to a one-hundred-twenty-third aspect, in any of the thirty-sixth to the thirty-eighth, the forty-second to the forty-fifth, the forty-seventh to the forty-ninth, or the fifty-first to the fifty-third aspects, the video receiving device has the same function of coding to transmit videos as the video coding device does, and the bit number received without errors for the constant period of time, which is the result of, as each of the unit time t equal to the least common multiple of the video frame period of the coded videos and the one frame time of the transmitted frames passes, accumulating the bit number received without errors for the unit time t between the constant period of time before the present time and the present time, is transmitted to the video coding device as each of the unit time t passes.
As stated above, according to the one-hundred-twenty-third aspect, taking the unit time t as the least common multiple of the transmission frame time and the period of videos easily enables synchronization with both of them, and by transmitting the bit number received without errors for the constant period of time for each of the unit time t to the video coding device, the average throughput can be obtained for each of the unit time t and careful control can be realized in the video coding device
According to a one-hundred-twenty-four th aspect, in any of the thirty-ninth, the forty-sixth, the fiftieth, the fifty-fourth, the sixty-fifth to seventy-first, or the one-hundred-seventh to the one-hundred-eighth aspects, the video receiving device has the same function of coding to transmit videos as the video coding device does, and, as each of the unit time t equal to the least common multiple of the video frame period of the coded videos and the one frame time of the transmitted frames passes, by dividing by the constant period of time the bit number received without errors for the constant period of time, which is the result of accumulating the bit number received without errors for the unit time t between the constant period of time before the present time and the present time, the average throughput for the constant period of time is sent to the video coding device as each of the unit time t passes.
As stated above, according to the one-hundred-twenty-fourth aspect, taking the unit time t as the least common multiple of the transmission frame period and the period of videos easily enables synchronization with both of them, and transmitting the average throughput for the constant period of time to the video coding device for each of the unit time t enables careful control in the video coding device.
A one-hundred-twenty-fifth aspect of the present invention is directed to a data transmission system in which a coding device for coding to transmit data and a receiving device for applying certain processing to received data are connected so that they can communicate,
wherein the receiving device transmits the bit number received without errors for the constant period of time T, which is the result of, as each of a predetermined constant period of time T passes, accumulating the bit number received without errors for a predetermined unit time t for the constant period of time T, to the coding device as each of the time T passes, and the coding device calculates the average throughput for the constant period of time T on the basis of the bit number received from the receiving device.
As stated above, according to the one-hundred-twenty-fifth aspect, the bit number received without errors for the constant period of time T can be calculated with simple processing, and the average throughput for each of the constant period of time T can be obtained in the coding device.
A one-hundred-twenty-sixth aspect of the present invention is directed to a data transmission system in which a coding device for coding to transmit data and a receiving device for applying certain processing to received data are connected so that they can communicate,
wherein as a predetermined constant period of time T passes, by dividing by the constant period of time T the bit number received without errors for the constant period of time T, which is the result of accumulating the bit number received without errors for a predetermined unit time t for the constant period of time T, the receiving device calculates the average throughput for the constant period of time T as the predetermined constant period of time T passes.
As stated above, according to the one-hundred-twenty-sixth aspect, the average throughput for the constant period of time T can be calculated with simple processing.
A one-hundred-twenty-seventh aspect of the present invention is directed to a data transmission system in which a coding device for coding and transmitting data and a receiving device for applying certain processing to received data are connected so that they can communicate,
wherein the receiving device transmits the bit number received without errors for a predetermined constant period of time T, which is the result of as each of the predetermined unit time t passes, accumulating the bit number received without errors for the unit time t between the predetermined constant period of time T before a present time and the present time, to the coding device as each of the predetermined unit time t passes, and the coding device calculates the average throughput for the constant period of time T on the basis of the bit number received from the receiving device.
As stated above, according to the one-hundred-twenty-seventh aspect, by transmitting the bit number received without errors for the constant period of time T to the coding device for each of the unit time t, the average throughput for each of the unit time t can be obtained in the coding device.
A one-hundred-twenty-eighth aspect of the present invention is directed to a data transmission system in which a coding device for coding to transmit data and a receiving device for applying certain processing to received data are connected so that they can communicate,
wherein the receiving device divides as each of a predetermined unit time t passes, by a predetermined constant period of time T the bit number received without errors for the constant period of time T a, which is the result of accumulating the bit number received without errors for the unit time t between the predetermined constant period of time T before a present time and the present time, to calculate the average throughput for the constant period of time T as each of the unit time t passes.
As stated above, according to the one-hundred-twenty-eighth aspect, the average throughput for the constant period of time T can be calculated for each of the unit time t.
According to a one-hundred-twenty-ninth aspect, in any of the one-hundred-twenty-fifth to the one-hundred-twenty-eighth aspects, the unit time t is taken as a one frame time of the transmitted frames.
As stated above, in the one-hundred-twenty-ninth aspect, taking the unit time t as the data frame period of the coded data permits synchronization with the coding timing.
According to a one-hundred-thirtieth aspect, in any of the one-hundred-twenty-fifth to the one-hundred-twenty-eighth aspects, the receiving device has the same function of coding to transmit data as the coding device does, and takes the unit time t as the data frame period of the coded data.
As stated above, according to the one-hundred-thirtieth aspect, taking the unit time t as the one frame time of the transmit ted frames easily enables synchronization with the transmitting timing of the transmitted frames.
According to a one-hundred-thirty-first aspect, in any of the one-hundred-twenty-fifth to one-hundred-twenty-eighth aspects, the receiving device has the same function of coding to transmit data as the coding device does, and taking the unit time t as the least common multiple of the data frame period of the coded data and the one frame time of the transmitted frames.
As stated above, according to the one-hundred-thirty-first aspect, taking the unit time t as the least common multiplex of the transmission frame time and the period of data easily enables synchronization with both of them.
According to a one-hundred-thirty-second aspect, in the thirty-ninth aspect, when the video receiving device calculates the average throughput, the time for averaging is variable according to whether a burst error occurs or not.
As stated above, according to the one-hundred-thirty-second aspect, decreasing the time over which the average is obtained permits the quantization control quickly responsive to a change in throughput, and in the condition where burst type errors occur, setting the time over which the average is obtained larger than the cycle of the occurrence of the burst errors allows the quantization control to keep stable even with burst errors.
According to a one-hundred-thirty-third aspect, in the forty-sixth aspect, when the video receiving device calculates the average throughput, the averaging time is variable according to whether burst errors occur or not.
As stated above, according to the one-hundred-thirty-third aspect, decreasing the time over which the average is obtained permits the frame dropping control quickly responsive to a change in throughput, and in the condition where burst type errors occur, setting the time over which the average is obtained larger than the cycle of the occurrence of the burst errors allows the frame dropping control to keep stable even with burst errors.
According to a one-hundred-thirty-fourth aspect, in the fiftieth aspect, when the video receiving device calculates the average throughput, the time for averaging is variable according to whether burst errors occur or not.
As stated above, according to the one-hundred-thirty-fourth aspect, decreasing the time over which the average is obtained permits the quantization control and the frame dropping control quickly responsive to a change in throughput, and in the condition where burst type errors occur, setting the time over which the average is obtained larger than the cycle of the occurrence of the burst errors allows the quantization control to keep stable even with burst errors.
According to a one-hundred-thirty-fifth aspect, in the fifty-fourth aspect, when the video receiving device calculates the average throughput, the time for averaging is variable according to whether burst errors occur or not.
As stated above, according to the one-hundred-thirty-fifth aspect, decreasing the time over which the average is obtained permits the quantization control and the frame dropping control quickly responsive to a change in throughput, and in the condition where burst type errors occur, setting the time over which the average is obtained larger than the cycle of the occurrence of the burst errors allows the quantization control and the frame dropping to keep stable even with burst errors.
According to a one-hundred-thirty-sixth aspect, in any of the one-hundred-thirty-second to the one-hundred-thirty-fifth, a period in which the bit number received without errors for a predetermined unit time t is smaller than a predetermined threshold is detected, and taken as the averaging time for use in calculating the average throughput.
As stated above, according to the one-hundred-thirty-sixth aspect, taking the burst error cycle as the averaging time for use in obtaining the average throughput permits the stable quantization control and the frame dropping control toward burst errors.
According to a one-hundred-thirty-seventh aspect, in any of the one-hundred-thirty-second to one-hundred thirty-fifth, the period in which the bit number received without errors for the unit time t equal to the one frame time of the transmitted frames is smaller than the predetermined threshold is detected, and taken as the averaging time used for calculating the average throughput.
As stated above, according to the one-hundred-thirty-seventh aspect, taking the burst error cycle as the averaging time used for obtaining the average throughput permits the stable quantization control and the frame dropping control toward burst errors.
According to a one-hundred-thirty-eighth aspect, in any of the one-hundred-thirty-second to the one-hundred-thirty-fifth aspects, the video receiving device has the same function of coding and transmitting videos as the video coding device does, the period in which the bit number received without errors for the unit time t equal to the video frame period of the coded videos is smaller than the predetermined threshold is detected, and taken as the averaging time used for calculating the average throughput.
As stated above, according to the one-hundred-thirty-eighth aspect, taking the burst error cycle as the averaging time used for obtaining the average throughput permits the stable quantization control and the frame dropping control toward burst errors.
According to a one-hundred-thirty-ninth, in any of the one-hundred-thirty-second to the one-hundred-thirty-fifth aspects, the video receiving device has the same function of coding to transmit videos as the video coding device does, the period in which the bit number received without errors for the unit time t equal to a least common multiplex between the video frame period of the coded videos and the one frame time of the transmitted frames is smaller than the predetermined threshold is detected, and taken as the averaging time used for calculating the average throughput.
As stated above, according to the one-hundred-thirty-ninth aspect, taking the burst error cycle as the averaging time used for obtaining the average throughput permits the stable quantization control and the frame dropping control toward burst errors.